Jul 12 22:08:58.056796 ip-10-0-141-188 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 12 22:08:58.056805 ip-10-0-141-188 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 12 22:08:58.056812 ip-10-0-141-188 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 12 22:08:58.057042 ip-10-0-141-188 systemd[1]: Failed to start Kubernetes Kubelet. Jul 12 22:09:08.243413 ip-10-0-141-188 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 12 22:09:08.243431 ip-10-0-141-188 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 687465afb627426d80aa6f0f5294ab5a -- Jul 12 22:11:03.118532 ip-10-0-141-188 systemd[1]: Starting Kubernetes Kubelet... Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.539176 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544830 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544844 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544848 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544852 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544855 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544859 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544863 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:11:03.609225 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544866 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544869 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544871 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544874 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544877 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544880 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544883 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544886 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544895 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544898 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544901 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544903 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544908 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544912 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544916 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544919 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544923 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544926 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544929 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:11:03.610603 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544932 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544936 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544939 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544943 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544946 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544950 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544953 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544956 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544959 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544962 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544965 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544968 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544971 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544974 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544976 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544979 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544982 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544985 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544988 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544991 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544994 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:11:03.611274 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.544997 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545000 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545002 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545005 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545009 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545013 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545015 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545018 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545021 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545026 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545029 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545032 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545036 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545041 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545044 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545047 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545050 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545053 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545056 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:11:03.612810 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545059 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545062 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545065 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545068 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545071 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545075 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545078 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545081 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545084 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545087 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545091 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545094 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545098 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545101 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545103 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545106 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545109 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545111 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545114 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545117 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:11:03.613408 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545119 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545122 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545125 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545128 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545130 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545133 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545136 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545139 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545142 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545144 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545149 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545153 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545155 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545948 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545956 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545959 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545961 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545964 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545967 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:11:03.614039 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545971 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545973 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545976 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545981 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545984 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545988 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545991 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545993 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545996 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.545999 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546002 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546005 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546008 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546010 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546013 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546016 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546018 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546022 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546024 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546027 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:11:03.614643 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546030 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546033 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546036 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546038 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546041 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546044 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546047 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546052 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546055 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546058 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546060 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546063 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546069 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546073 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546076 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546080 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546084 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546087 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546090 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:11:03.615364 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546093 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546098 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546101 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546105 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546108 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546111 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546115 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546117 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546120 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546123 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546125 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546128 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546131 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546134 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546137 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546139 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546142 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546145 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546147 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:11:03.616399 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546154 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546157 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546160 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546162 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546165 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546168 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546171 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546174 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546177 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546181 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546184 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546189 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546192 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546194 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546197 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546200 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546203 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546206 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546209 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:11:03.617170 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546212 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546255 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546278 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546283 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546287 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546291 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546295 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546300 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546303 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546306 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546309 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546313 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546317 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546320 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546324 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546327 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546776 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546783 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546787 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546790 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:11:03.617719 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546793 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546796 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546799 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546802 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546805 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546807 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546810 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546813 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546816 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546819 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546822 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546824 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546827 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546832 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546835 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546837 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546841 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546844 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546847 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546849 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:11:03.618250 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546852 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546855 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546858 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546861 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546863 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546866 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546868 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546871 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546874 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546876 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546879 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546882 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546885 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546888 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546890 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546893 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546896 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546901 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546904 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546907 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:11:03.618813 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546910 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546913 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546916 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546918 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546924 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546927 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546930 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546933 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546936 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546939 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546942 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546944 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546947 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546950 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546953 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546958 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546963 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546966 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546969 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:11:03.619360 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546972 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546976 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546981 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546984 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546987 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546990 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546993 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546996 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.546998 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547001 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547004 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547006 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547009 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547012 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547014 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547017 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547020 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547024 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547029 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:11:03.619931 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547032 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547035 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547038 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547041 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547044 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547047 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547050 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547052 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547055 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547058 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547060 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547063 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547066 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547068 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547071 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547074 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547076 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547149 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547157 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547174 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547179 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 22:11:03.620451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547184 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547188 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547192 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547197 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547200 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547204 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547207 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547211 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547214 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547219 2579 flags.go:64] FLAG: --cgroup-root="" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547223 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547227 2579 flags.go:64] FLAG: --client-ca-file="" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547230 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547234 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547238 2579 flags.go:64] FLAG: --cluster-domain="" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547241 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547245 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547249 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547252 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547257 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547260 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547263 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547266 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547270 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 12 22:11:03.621108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547273 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547277 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547280 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547283 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547287 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547291 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547294 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547297 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547300 2579 flags.go:64] FLAG: --enable-server="true" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547303 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547308 2579 flags.go:64] FLAG: --event-burst="100" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547312 2579 flags.go:64] FLAG: --event-qps="50" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547315 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547332 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547335 2579 flags.go:64] FLAG: --eviction-hard="" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547340 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547343 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547348 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547352 2579 flags.go:64] FLAG: --eviction-soft="" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547356 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547359 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547362 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547366 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547369 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547372 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 22:11:03.621787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547375 2579 flags.go:64] FLAG: --feature-gates="" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547379 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547383 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547386 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547389 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547393 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547396 2579 flags.go:64] FLAG: --help="false" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547399 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547402 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547405 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547409 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547412 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547416 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547419 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547422 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547426 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547429 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547432 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547435 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547438 2579 flags.go:64] FLAG: --kube-reserved="" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547442 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547445 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547448 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547451 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 22:11:03.622530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547456 2579 flags.go:64] FLAG: --lock-file="" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547460 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547464 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547467 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547473 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547477 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547480 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547483 2579 flags.go:64] FLAG: --logging-format="text" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547486 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547490 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547493 2579 flags.go:64] FLAG: --manifest-url="" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547496 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547501 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547504 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547509 2579 flags.go:64] FLAG: --max-pods="110" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547512 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547515 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547518 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547521 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547524 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547527 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547531 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547538 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547542 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547545 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 22:11:03.623204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547548 2579 flags.go:64] FLAG: --pod-cidr="" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547551 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547557 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547560 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547563 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547567 2579 flags.go:64] FLAG: --port="10250" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547570 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547574 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-041e988640c718c3a" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547578 2579 flags.go:64] FLAG: --qos-reserved="" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547583 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547586 2579 flags.go:64] FLAG: --register-node="true" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547589 2579 flags.go:64] FLAG: --register-with-taints="" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547593 2579 flags.go:64] FLAG: --registry-burst="10" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547596 2579 flags.go:64] FLAG: --registry-qps="5" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547600 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547603 2579 flags.go:64] FLAG: --reserved-memory="" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547607 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547611 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547614 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547617 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547621 2579 flags.go:64] FLAG: --runonce="false" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547624 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547627 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547631 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547634 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 22:11:03.624230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547637 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547640 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547643 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547646 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547649 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547652 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547655 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547658 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547662 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547665 2579 flags.go:64] FLAG: --system-cgroups="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547668 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547673 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547676 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547679 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547685 2579 flags.go:64] FLAG: --tls-min-version="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547688 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547693 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547696 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547699 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547703 2579 flags.go:64] FLAG: --v="2" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547707 2579 flags.go:64] FLAG: --version="false" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547711 2579 flags.go:64] FLAG: --vmodule="" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547716 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.547719 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547838 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:11:03.625010 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547842 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547846 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547850 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547853 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547857 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547860 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547862 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547865 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547868 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547871 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547875 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547879 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547882 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547884 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547887 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547891 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547895 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547898 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547901 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:11:03.625689 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547904 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547907 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547911 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547914 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547918 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547921 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547924 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547927 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547930 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547933 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547935 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547939 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547941 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547944 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547947 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547950 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547953 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547955 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547958 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:11:03.626213 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547961 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547979 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547983 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547986 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547990 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547993 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547996 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.547999 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548002 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548005 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548008 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548010 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548013 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548016 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548019 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548023 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548026 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548031 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548034 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548037 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:11:03.626837 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548039 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548042 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548045 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548048 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548051 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548054 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548057 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548060 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548063 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548066 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548068 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548071 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548074 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548076 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548079 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548082 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548085 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548088 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548090 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548093 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:11:03.627448 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548095 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548098 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548101 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548104 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548107 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548110 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548112 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548116 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548119 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548123 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548126 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548128 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548132 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548135 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548138 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548141 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548144 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548146 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548150 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:11:03.628008 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.548152 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.548158 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.556994 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.557010 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.557983 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.557991 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558065 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558071 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558074 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558078 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558082 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558085 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558089 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558092 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558095 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:11:03.628517 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558098 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558101 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558104 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558107 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558110 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558113 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558116 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558118 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558121 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558124 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558127 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558136 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558139 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558142 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558145 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558149 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558151 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558154 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558157 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558160 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:11:03.628971 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558163 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558166 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558168 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558171 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558174 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558177 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558180 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558184 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558188 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558191 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558195 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558198 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558201 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558204 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558207 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558210 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558214 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558217 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558220 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:11:03.629519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558223 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558226 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558229 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558232 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558235 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558244 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558248 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558252 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558255 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558272 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558275 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558284 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558287 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558290 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558293 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558296 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558299 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558302 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558304 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558307 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:11:03.630005 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558310 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558314 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558318 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558323 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558327 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558331 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558334 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558337 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558340 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558343 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558346 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558349 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558351 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558354 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558357 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558360 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558363 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558366 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558375 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558380 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:11:03.630546 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558385 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558388 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558390 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558393 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558396 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558400 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558405 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558410 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558413 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558416 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558420 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.558425 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558587 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558594 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:11:03.631093 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558597 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:11:03.630737 ip-10-0-141-188 systemd[1]: Started Kubernetes Kubelet. Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558600 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558603 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558606 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558609 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558612 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558614 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558617 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558620 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558622 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558625 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558628 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558630 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558633 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558636 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558640 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558643 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558651 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558661 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:11:03.631674 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558664 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558667 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558669 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558672 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558675 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558678 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558681 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558683 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558686 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558689 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558692 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558694 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558699 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558702 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558706 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558709 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558712 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558715 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558718 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:11:03.632244 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558721 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558724 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558727 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558730 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558733 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558736 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558738 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558741 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558744 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558746 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558749 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558752 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558764 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558767 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558770 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558773 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558776 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558778 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558781 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558784 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:11:03.632816 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558787 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558789 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558792 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558795 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558798 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558800 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558803 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558806 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558809 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558811 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558814 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558817 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558819 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558822 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558824 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558827 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558830 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558832 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558835 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558837 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:11:03.633407 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558840 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558843 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558847 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558851 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558857 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558866 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558869 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558872 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558874 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558877 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558880 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558883 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558885 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558888 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558891 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558894 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558897 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558899 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:11:03.633966 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:03.558902 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.558908 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.559026 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.561266 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.562546 2579 server.go:1019] "Starting client certificate rotation" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.562657 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.562693 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.565233 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.566715 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.587335 2579 log.go:25] "Validated CRI v1 runtime API" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.594684 2579 log.go:25] "Validated CRI v1 image API" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.595687 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 22:11:03.634438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.599101 2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8161f01b-2780-40eb-b458-ec80eb2f62ba:/dev/nvme0n1p3 cc10e06a-facc-4d79-b9e0-c2908cf59e0f:/dev/nvme0n1p4] Jul 12 22:11:03.634768 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.599119 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 22:11:03.634768 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.601486 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.605613 2579 manager.go:216] Machine: {Timestamp:2026-07-12 22:11:03.603121161 +0000 UTC m=+0.357374412 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3084703 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c81938fadb0a8885fb362966a88a3 SystemUUID:ec2c8193-8fad-b0a8-885f-b362966a88a3 BootID:687465af-b627-426d-80aa-6f0f5294ab5a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9b:0f:ec:3c:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9b:0f:ec:3c:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:26:1c:62:2b:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.605729 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.605822 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.609511 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.609553 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.610428 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.610443 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.610453 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.610478 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.611840 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.615896 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616064 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616109 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616124 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616148 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616162 2579 kubelet.go:398] "Adding apiserver pod source" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616189 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 22:11:03.634833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616779 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.616805 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.621768 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.623743 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.623785 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624006 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624023 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624032 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624042 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624053 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624061 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624664 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624675 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624687 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624696 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624708 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.624720 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.625507 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.625517 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.627550 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.627682 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.627733 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.629684 2579 server.go:1268] "Started kubelet" Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.630719 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 22:11:03.635359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.632962 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 12 22:11:03.636301 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.635606 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 22:11:03.636301 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.635713 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 22:11:03.636930 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.636887 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xfdv7" Jul 12 22:11:03.637808 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.637787 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 22:11:03.640212 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.640194 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 22:11:03.640479 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.640451 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:03.640554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.640500 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 22:11:03.641241 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.641223 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 22:11:03.641367 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.641256 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 22:11:03.641367 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.641267 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 22:11:03.641477 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.641469 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 12 22:11:03.641527 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.641479 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 12 22:11:03.642273 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642236 2579 factory.go:153] Registering CRI-O factory Jul 12 22:11:03.642352 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642297 2579 factory.go:223] Registration of the crio container factory successfully Jul 12 22:11:03.642396 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642358 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 22:11:03.642396 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642369 2579 factory.go:55] Registering systemd factory Jul 12 22:11:03.642396 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642378 2579 factory.go:223] Registration of the systemd container factory successfully Jul 12 22:11:03.642513 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642402 2579 factory.go:103] Registering Raw factory Jul 12 22:11:03.642513 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.642423 2579 manager.go:1196] Started watching for new ooms in manager Jul 12 22:11:03.644232 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.643201 2579 manager.go:349] Starting recovery of all containers Jul 12 22:11:03.647429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.647406 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xfdv7" Jul 12 22:11:03.652726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.652692 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 22:11:03.655060 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.654890 2579 manager.go:354] Recovery completed Jul 12 22:11:03.655451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.655431 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:03.659337 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.659300 2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-188.ec2.internal\" not found" node="ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.661428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.661415 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.663509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.663494 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.663592 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.663523 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.663592 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.663533 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.664153 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.664139 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 22:11:03.664153 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.664150 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 22:11:03.664258 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.664153 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.664258 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.664168 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 12 22:11:03.665019 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.665003 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.665068 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.665030 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.665068 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.665046 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.666253 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.666241 2579 policy_none.go:49] "None policy: Start" Jul 12 22:11:03.666300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.666257 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 22:11:03.666300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.666268 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 22:11:03.668361 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.668346 2579 policy_none.go:47] "Start" Jul 12 22:11:03.702942 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.702924 2579 manager.go:337] "Starting Device Plugin manager" Jul 12 22:11:03.703070 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.702966 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 22:11:03.703070 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.702982 2579 server.go:87] "Starting device plugin registration server" Jul 12 22:11:03.703251 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.703237 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 22:11:03.703311 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.703255 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 22:11:03.703405 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.703386 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 22:11:03.703600 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.703477 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 22:11:03.703600 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.703485 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 22:11:03.704101 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.704081 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 22:11:03.704185 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.704141 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:03.803865 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.803834 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.805815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.805128 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.805815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.805161 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.805815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.805173 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.805815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.805202 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.816298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.816276 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.816421 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.816304 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-188.ec2.internal\": node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:03.846545 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.846524 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:03.850086 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.850058 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 22:11:03.850178 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.850103 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 22:11:03.850178 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.850127 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 22:11:03.850246 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.850166 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 22:11:03.852500 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.852480 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:03.947056 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.946943 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:03.950307 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.950281 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal"] Jul 12 22:11:03.950386 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.950355 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.951377 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.951358 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.951499 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.951388 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.951499 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.951400 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.953633 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.953619 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.953791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.953776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.953828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.953809 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.954400 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.954378 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.954511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.954414 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.954511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.954426 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.954511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.954487 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.954617 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.954517 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.954617 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.954529 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.956840 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.956826 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.956890 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.956859 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:11:03.957595 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.957579 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:11:03.957687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.957609 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:11:03.957687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:03.957623 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:11:03.978057 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.978037 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-188.ec2.internal\" not found" node="ip-10-0-141-188.ec2.internal" Jul 12 22:11:03.981822 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:03.981804 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-188.ec2.internal\" not found" node="ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.047117 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.047087 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.047287 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.047121 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0ad369be1433261009ab42f4c9aad4fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal\" (UID: \"0ad369be1433261009ab42f4c9aad4fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.047287 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.047147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad369be1433261009ab42f4c9aad4fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal\" (UID: \"0ad369be1433261009ab42f4c9aad4fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.047287 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.047165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81f992adf125e091bbf7fa1071ef9694-config\") pod \"kube-apiserver-proxy-ip-10-0-141-188.ec2.internal\" (UID: \"81f992adf125e091bbf7fa1071ef9694\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.147587 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.147548 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.147587 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.147574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81f992adf125e091bbf7fa1071ef9694-config\") pod \"kube-apiserver-proxy-ip-10-0-141-188.ec2.internal\" (UID: \"81f992adf125e091bbf7fa1071ef9694\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.147834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.147612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0ad369be1433261009ab42f4c9aad4fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal\" (UID: \"0ad369be1433261009ab42f4c9aad4fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.147834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.147639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad369be1433261009ab42f4c9aad4fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal\" (UID: \"0ad369be1433261009ab42f4c9aad4fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.147834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.147687 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0ad369be1433261009ab42f4c9aad4fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal\" (UID: \"0ad369be1433261009ab42f4c9aad4fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.147834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.147688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81f992adf125e091bbf7fa1071ef9694-config\") pod \"kube-apiserver-proxy-ip-10-0-141-188.ec2.internal\" (UID: \"81f992adf125e091bbf7fa1071ef9694\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.147834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.147732 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad369be1433261009ab42f4c9aad4fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal\" (UID: \"0ad369be1433261009ab42f4c9aad4fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.248055 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.247968 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.281687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.281657 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.284969 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.284947 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.348350 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.348307 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.448903 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.448877 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.469381 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.469349 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:04.549095 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.549009 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.562154 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.562121 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 22:11:04.562341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.562308 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 22:11:04.562341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.562327 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 22:11:04.562341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.562338 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 22:11:04.649747 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:04.649710 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-188.ec2.internal\" not found" Jul 12 22:11:04.652061 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.652036 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 22:06:04 +0000 UTC" deadline="2028-02-08 19:34:28.529107771 +0000 UTC" Jul 12 22:11:04.652127 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.652061 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13821h23m23.877049773s" Jul 12 22:11:04.694721 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.694694 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:04.742890 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.742840 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.757501 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.757477 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 22:11:04.758549 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.758533 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" Jul 12 22:11:04.768025 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:04.768006 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 22:11:05.011142 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:05.011108 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad369be1433261009ab42f4c9aad4fc.slice/crio-4f78f2aa767294fa53a2dc05c5297112f668b33bbe2e25133981e2c88a0b3423 WatchSource:0}: Error finding container 4f78f2aa767294fa53a2dc05c5297112f668b33bbe2e25133981e2c88a0b3423: Status 404 returned error can't find the container with id 4f78f2aa767294fa53a2dc05c5297112f668b33bbe2e25133981e2c88a0b3423 Jul 12 22:11:05.016005 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.015980 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:11:05.017498 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:05.017469 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f992adf125e091bbf7fa1071ef9694.slice/crio-24dcfa4d394e9087d63307d0ee180ebc91e631ee167df73e2b5b4a9a326b8ae5 WatchSource:0}: Error finding container 24dcfa4d394e9087d63307d0ee180ebc91e631ee167df73e2b5b4a9a326b8ae5: Status 404 returned error can't find the container with id 24dcfa4d394e9087d63307d0ee180ebc91e631ee167df73e2b5b4a9a326b8ae5 Jul 12 22:11:05.576061 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.576030 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:05.617601 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.617566 2579 apiserver.go:52] "Watching apiserver" Jul 12 22:11:05.627630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.627600 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 22:11:05.628227 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.628201 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-7c5q2","openshift-network-operator/iptables-alerter-scjjk","kube-system/konnectivity-agent-dd2q2","openshift-cluster-node-tuning-operator/tuned-76dp8","openshift-dns/node-resolver-jtvzw","openshift-image-registry/node-ca-tbnjt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal","openshift-multus/multus-additional-cni-plugins-2fbdv","openshift-network-diagnostics/network-check-target-8lxs2","openshift-ovn-kubernetes/ovnkube-node-n9zg9","kube-system/global-pull-secret-syncer-nj7st","kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq","openshift-multus/multus-rtcqm"] Jul 12 22:11:05.632613 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.631415 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.636074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.635437 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 22:11:05.636074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.635683 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 22:11:05.636074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.635744 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.636074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.635789 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.636074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.635808 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 22:11:05.636074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.635760 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 22:11:05.636498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.636109 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.636498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.636353 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8sjhs\"" Jul 12 22:11:05.638392 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.637980 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.639023 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.639000 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t65pq\"" Jul 12 22:11:05.639346 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.639282 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 22:11:05.639618 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.639598 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.639696 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.639666 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.640709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.640690 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 22:11:05.640835 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.640815 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qhtpw\"" Jul 12 22:11:05.640926 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.640693 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 22:11:05.641103 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.641083 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.641839 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.641818 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 22:11:05.643530 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.643509 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.644135 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.644113 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-69srw\"" Jul 12 22:11:05.644262 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.644239 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.644356 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.644338 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.646000 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.645982 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.646148 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.646126 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.646792 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.646761 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pbwjv\"" Jul 12 22:11:05.646893 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.646806 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.648315 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.648287 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.649686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.649667 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:05.649831 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.649802 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:05.650502 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.650270 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 22:11:05.650609 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.650591 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.650836 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.650822 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zcx4x\"" Jul 12 22:11:05.653208 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.653187 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.654314 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654291 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 22:11:05.654566 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654545 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-modprobe-d\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.654642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654582 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-sys\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.654642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aba800a0-daee-4a0c-a746-f4bf1159460c-tmp\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.654642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.654805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.654805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654695 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-socket-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.654805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-sys-fs\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.654805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysctl-conf\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-tuned\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654839 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-run-ovn-kubernetes\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-cni-bin\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/165d9e3d-8328-4146-b17a-c3b9348a411c-agent-certs\") pod \"konnectivity-agent-dd2q2\" (UID: \"165d9e3d-8328-4146-b17a-c3b9348a411c\") " pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654944 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-host\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654966 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qbtz\" (UniqueName: \"kubernetes.io/projected/aba800a0-daee-4a0c-a746-f4bf1159460c-kube-api-access-5qbtz\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.654989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjrgv\" (UniqueName: \"kubernetes.io/projected/6750b24d-b4dc-4842-b0a8-91d90c255345-kube-api-access-vjrgv\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-run-netns\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-cni-netd\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655099 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9pf\" (UniqueName: \"kubernetes.io/projected/edbb9cc6-2861-494e-b593-f8a0cad404ea-kube-api-access-dm9pf\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655124 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-systemd\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-kubelet\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-systemd-units\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655193 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-node-log\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovnkube-config\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655253 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovn-node-metrics-cert\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655292 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-var-lib-kubelet\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6750b24d-b4dc-4842-b0a8-91d90c255345-hosts-file\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-systemd\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655413 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-var-lib-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655440 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-etc-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-log-socket\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655503 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysconfig\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655528 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-kubernetes\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysctl-d\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655675 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6750b24d-b4dc-4842-b0a8-91d90c255345-tmp-dir\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-ovn\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-device-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655768 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wsqn\" (UniqueName: \"kubernetes.io/projected/1f79d4b2-118f-4114-9543-04840dedd0c7-kube-api-access-6wsqn\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655802 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-lib-modules\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-env-overrides\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/165d9e3d-8328-4146-b17a-c3b9348a411c-konnectivity-ca\") pod \"konnectivity-agent-dd2q2\" (UID: \"165d9e3d-8328-4146-b17a-c3b9348a411c\") " pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-run\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.655904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655908 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-slash\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.657071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovnkube-script-lib\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.657071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-registration-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.657071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.655985 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.657071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.656561 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.657071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.656796 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.657690 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.657669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.657803 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.657734 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 22:11:05.658236 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.658216 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pk4nv\"" Jul 12 22:11:05.658482 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.658464 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 22:11:05.661731 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.661446 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:05.661731 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.661531 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:05.663288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.663115 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.663288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.663177 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 22:11:05.663819 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.663616 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pm2bv\"" Jul 12 22:11:05.663819 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.663627 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.664279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.664240 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.664386 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.664340 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:05.667153 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.667091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.674926 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.674896 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8k662\"" Jul 12 22:11:05.675195 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.675177 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 22:11:05.675305 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.675289 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 22:11:05.686935 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.686906 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mntrw" Jul 12 22:11:05.704752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.704597 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mntrw" Jul 12 22:11:05.742469 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.742429 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 22:11:05.756349 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6750b24d-b4dc-4842-b0a8-91d90c255345-hosts-file\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.756520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-log-socket\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.756520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-cni-binary-copy\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.756520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756415 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-conf-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.756520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756443 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.756520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756461 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6750b24d-b4dc-4842-b0a8-91d90c255345-hosts-file\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.756520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45091bed-7fab-448b-94e6-1b98df48245b-iptables-alerter-script\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-kubernetes\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756545 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-log-socket\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756566 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysctl-d\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6750b24d-b4dc-4842-b0a8-91d90c255345-tmp-dir\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-ovn\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756718 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-device-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wsqn\" (UniqueName: \"kubernetes.io/projected/1f79d4b2-118f-4114-9543-04840dedd0c7-kube-api-access-6wsqn\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysctl-d\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756794 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-os-release\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756802 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-kubernetes\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756822 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-kubelet\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756844 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-ovn\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.756902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-lib-modules\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756983 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6750b24d-b4dc-4842-b0a8-91d90c255345-tmp-dir\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.756991 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-lib-modules\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757045 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-device-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757075 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/165d9e3d-8328-4146-b17a-c3b9348a411c-konnectivity-ca\") pod \"konnectivity-agent-dd2q2\" (UID: \"165d9e3d-8328-4146-b17a-c3b9348a411c\") " pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7k6m\" (UniqueName: \"kubernetes.io/projected/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-kube-api-access-m7k6m\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-os-release\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757164 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-kubelet-config\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757232 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-run\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757257 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-slash\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757300 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757374 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-run\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-slash\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757415 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-serviceca\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.757584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-socket-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757515 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cnibin\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvmp7\" (UniqueName: \"kubernetes.io/projected/45091bed-7fab-448b-94e6-1b98df48245b-kube-api-access-zvmp7\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757600 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757655 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-tuned\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757727 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-socket-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757727 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/165d9e3d-8328-4146-b17a-c3b9348a411c-konnectivity-ca\") pod \"konnectivity-agent-dd2q2\" (UID: \"165d9e3d-8328-4146-b17a-c3b9348a411c\") " pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757722 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-run-ovn-kubernetes\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-cni-bin\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757868 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-netns\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757899 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-cni-multus\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-daemon-config\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.757958 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.758413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758000 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjrgv\" (UniqueName: \"kubernetes.io/projected/6750b24d-b4dc-4842-b0a8-91d90c255345-kube-api-access-vjrgv\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-run-ovn-kubernetes\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758037 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-cni-netd\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758054 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9pf\" (UniqueName: \"kubernetes.io/projected/edbb9cc6-2861-494e-b593-f8a0cad404ea-kube-api-access-dm9pf\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-hostroot\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758159 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-kubelet\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-systemd-units\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-node-log\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-system-cni-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-kubelet\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-socket-dir-parent\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758428 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-node-log\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758442 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-cni-bin\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758429 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-etc-kubernetes\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-system-cni-dir\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758504 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-cni-netd\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-var-lib-kubelet\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.759122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758569 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-systemd-units\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758572 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-systemd\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758616 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-run-systemd\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-var-lib-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-etc-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-dbus\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-var-lib-kubelet\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysconfig\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758761 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cni-binary-copy\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-var-lib-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758796 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-etc-openvswitch\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758809 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysconfig\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758845 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-env-overrides\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758876 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-cnibin\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovnkube-script-lib\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.758959 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-registration-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759079 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.760012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759113 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-multus-certs\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759155 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45091bed-7fab-448b-94e6-1b98df48245b-host-slash\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-modprobe-d\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-registration-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-sys\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759320 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759353 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-env-overrides\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759367 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-modprobe-d\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759369 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-sys\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759396 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aba800a0-daee-4a0c-a746-f4bf1159460c-tmp\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-sys-fs\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-cni-bin\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759481 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsn4s\" (UniqueName: \"kubernetes.io/projected/1cfdeee8-d3fc-4789-996f-0db8e40a4064-kube-api-access-hsn4s\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759491 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-sys-fs\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759506 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dff4q\" (UniqueName: \"kubernetes.io/projected/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-kube-api-access-dff4q\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759550 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovnkube-script-lib\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysctl-conf\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.760870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/165d9e3d-8328-4146-b17a-c3b9348a411c-agent-certs\") pod \"konnectivity-agent-dd2q2\" (UID: \"165d9e3d-8328-4146-b17a-c3b9348a411c\") " pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759666 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-k8s-cni-cncf-io\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759685 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-sysctl-conf\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-host\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-host\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759740 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f79d4b2-118f-4114-9543-04840dedd0c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759779 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-host\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759902 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbtz\" (UniqueName: \"kubernetes.io/projected/aba800a0-daee-4a0c-a746-f4bf1159460c-kube-api-access-5qbtz\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-run-netns\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.759989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgwjn\" (UniqueName: \"kubernetes.io/projected/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-kube-api-access-lgwjn\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-systemd\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovnkube-config\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760079 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovn-node-metrics-cert\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-cni-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760614 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovnkube-config\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760705 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edbb9cc6-2861-494e-b593-f8a0cad404ea-host-run-netns\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.761706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.760775 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-systemd\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.762521 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.761925 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aba800a0-daee-4a0c-a746-f4bf1159460c-etc-tuned\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.763096 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.763075 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/165d9e3d-8328-4146-b17a-c3b9348a411c-agent-certs\") pod \"konnectivity-agent-dd2q2\" (UID: \"165d9e3d-8328-4146-b17a-c3b9348a411c\") " pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.763096 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.763094 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edbb9cc6-2861-494e-b593-f8a0cad404ea-ovn-node-metrics-cert\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.763317 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.763127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aba800a0-daee-4a0c-a746-f4bf1159460c-tmp\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.766828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.766804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wsqn\" (UniqueName: \"kubernetes.io/projected/1f79d4b2-118f-4114-9543-04840dedd0c7-kube-api-access-6wsqn\") pod \"aws-ebs-csi-driver-node-dg7mq\" (UID: \"1f79d4b2-118f-4114-9543-04840dedd0c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.767503 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.767461 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9pf\" (UniqueName: \"kubernetes.io/projected/edbb9cc6-2861-494e-b593-f8a0cad404ea-kube-api-access-dm9pf\") pod \"ovnkube-node-n9zg9\" (UID: \"edbb9cc6-2861-494e-b593-f8a0cad404ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.769491 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.769449 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjrgv\" (UniqueName: \"kubernetes.io/projected/6750b24d-b4dc-4842-b0a8-91d90c255345-kube-api-access-vjrgv\") pod \"node-resolver-jtvzw\" (UID: \"6750b24d-b4dc-4842-b0a8-91d90c255345\") " pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.771189 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.771164 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qbtz\" (UniqueName: \"kubernetes.io/projected/aba800a0-daee-4a0c-a746-f4bf1159460c-kube-api-access-5qbtz\") pod \"tuned-76dp8\" (UID: \"aba800a0-daee-4a0c-a746-f4bf1159460c\") " pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.856596 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.856491 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" event={"ID":"81f992adf125e091bbf7fa1071ef9694","Type":"ContainerStarted","Data":"24dcfa4d394e9087d63307d0ee180ebc91e631ee167df73e2b5b4a9a326b8ae5"} Jul 12 22:11:05.858554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.858519 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" event={"ID":"0ad369be1433261009ab42f4c9aad4fc","Type":"ContainerStarted","Data":"4f78f2aa767294fa53a2dc05c5297112f668b33bbe2e25133981e2c88a0b3423"} Jul 12 22:11:05.861143 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-os-release\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-kubelet\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7k6m\" (UniqueName: \"kubernetes.io/projected/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-kube-api-access-m7k6m\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-os-release\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.861298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861285 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-kubelet-config\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861342 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-os-release\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861377 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-serviceca\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cnibin\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861419 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-os-release\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861435 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvmp7\" (UniqueName: \"kubernetes.io/projected/45091bed-7fab-448b-94e6-1b98df48245b-kube-api-access-zvmp7\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861342 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-kubelet\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cnibin\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861494 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-netns\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-cni-multus\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-netns\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861407 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-kubelet-config\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.861554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-daemon-config\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.861651 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-hostroot\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-system-cni-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.861742 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:06.361697894 +0000 UTC m=+3.115951132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861792 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-socket-dir-parent\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861824 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-etc-kubernetes\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-system-cni-dir\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-dbus\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cni-binary-copy\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861945 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-hostroot\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-cnibin\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-multus-certs\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861993 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-cnibin\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862026 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-cni-multus\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.862337 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.862029 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.861741 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-system-cni-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862064 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-system-cni-dir\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45091bed-7fab-448b-94e6-1b98df48245b-host-slash\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.862097 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:06.362078791 +0000 UTC m=+3.116332033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862134 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-socket-dir-parent\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-cni-bin\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45091bed-7fab-448b-94e6-1b98df48245b-host-slash\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsn4s\" (UniqueName: \"kubernetes.io/projected/1cfdeee8-d3fc-4789-996f-0db8e40a4064-kube-api-access-hsn4s\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dff4q\" (UniqueName: \"kubernetes.io/projected/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-kube-api-access-dff4q\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-daemon-config\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-multus-certs\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862312 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-dbus\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862333 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-etc-kubernetes\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862357 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-var-lib-cni-bin\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-k8s-cni-cncf-io\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.863230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-host\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-host-run-k8s-cni-cncf-io\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862634 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-host\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgwjn\" (UniqueName: \"kubernetes.io/projected/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-kube-api-access-lgwjn\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-cni-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1cfdeee8-d3fc-4789-996f-0db8e40a4064-cni-binary-copy\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-cni-binary-copy\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-conf-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862850 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-cni-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862882 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-multus-conf-dir\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862899 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45091bed-7fab-448b-94e6-1b98df48245b-iptables-alerter-script\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.862979 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-serviceca\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.863005 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1cfdeee8-d3fc-4789-996f-0db8e40a4064-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.863232 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-cni-binary-copy\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.864119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.863394 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45091bed-7fab-448b-94e6-1b98df48245b-iptables-alerter-script\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.872627 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.872591 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:05.872627 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.872619 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:05.872627 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.872634 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:05.872888 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:05.872707 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:06.372690702 +0000 UTC m=+3.126943941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:05.872989 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.872963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7k6m\" (UniqueName: \"kubernetes.io/projected/d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da-kube-api-access-m7k6m\") pod \"multus-rtcqm\" (UID: \"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da\") " pod="openshift-multus/multus-rtcqm" Jul 12 22:11:05.875362 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.875281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsn4s\" (UniqueName: \"kubernetes.io/projected/1cfdeee8-d3fc-4789-996f-0db8e40a4064-kube-api-access-hsn4s\") pod \"multus-additional-cni-plugins-2fbdv\" (UID: \"1cfdeee8-d3fc-4789-996f-0db8e40a4064\") " pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:05.875506 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.875483 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dff4q\" (UniqueName: \"kubernetes.io/projected/7a5ffff7-47c3-4aad-8d8a-9be8675d178e-kube-api-access-dff4q\") pod \"node-ca-tbnjt\" (UID: \"7a5ffff7-47c3-4aad-8d8a-9be8675d178e\") " pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:05.875813 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.875791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgwjn\" (UniqueName: \"kubernetes.io/projected/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-kube-api-access-lgwjn\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:05.878890 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.878866 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvmp7\" (UniqueName: \"kubernetes.io/projected/45091bed-7fab-448b-94e6-1b98df48245b-kube-api-access-zvmp7\") pod \"iptables-alerter-scjjk\" (UID: \"45091bed-7fab-448b-94e6-1b98df48245b\") " pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:05.943126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.943090 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:05.952574 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.952542 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:05.963091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.963062 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" Jul 12 22:11:05.965950 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:05.965550 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbb9cc6_2861_494e_b593_f8a0cad404ea.slice/crio-378993ebdcac1edeb3bf5662edf287cc8e22dcb68eb6c8090bf56a5e2f6c7e4c WatchSource:0}: Error finding container 378993ebdcac1edeb3bf5662edf287cc8e22dcb68eb6c8090bf56a5e2f6c7e4c: Status 404 returned error can't find the container with id 378993ebdcac1edeb3bf5662edf287cc8e22dcb68eb6c8090bf56a5e2f6c7e4c Jul 12 22:11:05.974535 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:05.974506 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f79d4b2_118f_4114_9543_04840dedd0c7.slice/crio-8403adde985e5f02b860e885c9bc1b68358459ccc9c45e3b191e0b8e25d17d5f WatchSource:0}: Error finding container 8403adde985e5f02b860e885c9bc1b68358459ccc9c45e3b191e0b8e25d17d5f: Status 404 returned error can't find the container with id 8403adde985e5f02b860e885c9bc1b68358459ccc9c45e3b191e0b8e25d17d5f Jul 12 22:11:05.978104 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.978079 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:05.988053 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.988029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-76dp8" Jul 12 22:11:05.989348 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:05.989318 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165d9e3d_8328_4146_b17a_c3b9348a411c.slice/crio-ad3a8449b8ba8201b919755c70d331d6a84e5a575733ee5664072c43c3d11353 WatchSource:0}: Error finding container ad3a8449b8ba8201b919755c70d331d6a84e5a575733ee5664072c43c3d11353: Status 404 returned error can't find the container with id ad3a8449b8ba8201b919755c70d331d6a84e5a575733ee5664072c43c3d11353 Jul 12 22:11:05.997193 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:05.997172 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jtvzw" Jul 12 22:11:05.999079 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:05.999045 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba800a0_daee_4a0c_a746_f4bf1159460c.slice/crio-0702952618ce73eda4e516302e062a08d3d54cada410b93a810e00c8705eda99 WatchSource:0}: Error finding container 0702952618ce73eda4e516302e062a08d3d54cada410b93a810e00c8705eda99: Status 404 returned error can't find the container with id 0702952618ce73eda4e516302e062a08d3d54cada410b93a810e00c8705eda99 Jul 12 22:11:06.007948 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.007923 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tbnjt" Jul 12 22:11:06.009238 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:06.009211 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6750b24d_b4dc_4842_b0a8_91d90c255345.slice/crio-b904a95a74cc2295585b995f2ca52dda2b709004f4286bb895800eb4d6721452 WatchSource:0}: Error finding container b904a95a74cc2295585b995f2ca52dda2b709004f4286bb895800eb4d6721452: Status 404 returned error can't find the container with id b904a95a74cc2295585b995f2ca52dda2b709004f4286bb895800eb4d6721452 Jul 12 22:11:06.016841 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.016818 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rtcqm" Jul 12 22:11:06.018973 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:06.018943 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5ffff7_47c3_4aad_8d8a_9be8675d178e.slice/crio-2db5a70038e7145c26532f4222eba74a6ec6a675e8f48ce709ecf45ceae742db WatchSource:0}: Error finding container 2db5a70038e7145c26532f4222eba74a6ec6a675e8f48ce709ecf45ceae742db: Status 404 returned error can't find the container with id 2db5a70038e7145c26532f4222eba74a6ec6a675e8f48ce709ecf45ceae742db Jul 12 22:11:06.025380 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.025358 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-scjjk" Jul 12 22:11:06.028297 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:06.028259 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a3127f_38ce_4b6d_a9a0_d4c0ec1830da.slice/crio-dade3c8c5f8ddea36436eca25beefe3b32f26a43c11ca512333356ac0cb28fc8 WatchSource:0}: Error finding container dade3c8c5f8ddea36436eca25beefe3b32f26a43c11ca512333356ac0cb28fc8: Status 404 returned error can't find the container with id dade3c8c5f8ddea36436eca25beefe3b32f26a43c11ca512333356ac0cb28fc8 Jul 12 22:11:06.032252 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.032227 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" Jul 12 22:11:06.036812 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:06.036787 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45091bed_7fab_448b_94e6_1b98df48245b.slice/crio-3d61401a4fdd963538d54d0eb9b55cc161424d76f94c0ecacd87324648ab8c26 WatchSource:0}: Error finding container 3d61401a4fdd963538d54d0eb9b55cc161424d76f94c0ecacd87324648ab8c26: Status 404 returned error can't find the container with id 3d61401a4fdd963538d54d0eb9b55cc161424d76f94c0ecacd87324648ab8c26 Jul 12 22:11:06.043805 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:06.043774 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfdeee8_d3fc_4789_996f_0db8e40a4064.slice/crio-b676bbcd2d42535eebfd51b1a314f1eba01618b7645b9e78c10a34ff8b1219e9 WatchSource:0}: Error finding container b676bbcd2d42535eebfd51b1a314f1eba01618b7645b9e78c10a34ff8b1219e9: Status 404 returned error can't find the container with id b676bbcd2d42535eebfd51b1a314f1eba01618b7645b9e78c10a34ff8b1219e9 Jul 12 22:11:06.366865 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.366823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:06.367065 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.366883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:06.367065 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.366994 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:06.367065 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.367057 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:07.367038503 +0000 UTC m=+4.121291740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:06.367232 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.366996 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:06.367232 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.367193 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:07.367159838 +0000 UTC m=+4.121413078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:06.468063 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.468027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:06.468285 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.468236 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:06.468285 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.468280 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:06.468413 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.468292 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:06.468413 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:06.468356 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:07.468335551 +0000 UTC m=+4.222588803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:06.707979 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.707873 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 22:06:06 +0000 UTC" deadline="2028-04-21 22:08:19.07755773 +0000 UTC" Jul 12 22:11:06.707979 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.707919 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15575h57m12.36964474s" Jul 12 22:11:06.863591 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.863557 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerStarted","Data":"b676bbcd2d42535eebfd51b1a314f1eba01618b7645b9e78c10a34ff8b1219e9"} Jul 12 22:11:06.865598 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.865567 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-scjjk" event={"ID":"45091bed-7fab-448b-94e6-1b98df48245b","Type":"ContainerStarted","Data":"3d61401a4fdd963538d54d0eb9b55cc161424d76f94c0ecacd87324648ab8c26"} Jul 12 22:11:06.867286 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.867237 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rtcqm" event={"ID":"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da","Type":"ContainerStarted","Data":"dade3c8c5f8ddea36436eca25beefe3b32f26a43c11ca512333356ac0cb28fc8"} Jul 12 22:11:06.868622 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.868584 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jtvzw" event={"ID":"6750b24d-b4dc-4842-b0a8-91d90c255345","Type":"ContainerStarted","Data":"b904a95a74cc2295585b995f2ca52dda2b709004f4286bb895800eb4d6721452"} Jul 12 22:11:06.869991 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.869967 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dd2q2" event={"ID":"165d9e3d-8328-4146-b17a-c3b9348a411c","Type":"ContainerStarted","Data":"ad3a8449b8ba8201b919755c70d331d6a84e5a575733ee5664072c43c3d11353"} Jul 12 22:11:06.871409 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.871358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"378993ebdcac1edeb3bf5662edf287cc8e22dcb68eb6c8090bf56a5e2f6c7e4c"} Jul 12 22:11:06.872525 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.872498 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tbnjt" event={"ID":"7a5ffff7-47c3-4aad-8d8a-9be8675d178e","Type":"ContainerStarted","Data":"2db5a70038e7145c26532f4222eba74a6ec6a675e8f48ce709ecf45ceae742db"} Jul 12 22:11:06.875695 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.875644 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-76dp8" event={"ID":"aba800a0-daee-4a0c-a746-f4bf1159460c","Type":"ContainerStarted","Data":"0702952618ce73eda4e516302e062a08d3d54cada410b93a810e00c8705eda99"} Jul 12 22:11:06.877283 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:06.877236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" event={"ID":"1f79d4b2-118f-4114-9543-04840dedd0c7","Type":"ContainerStarted","Data":"8403adde985e5f02b860e885c9bc1b68358459ccc9c45e3b191e0b8e25d17d5f"} Jul 12 22:11:07.000533 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.000370 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:11:07.375484 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.375227 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:07.375605 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.375517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:07.375605 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.375459 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:07.375721 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.375636 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:07.375721 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.375638 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:09.37561375 +0000 UTC m=+6.129867009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:07.375721 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.375697 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:09.375679716 +0000 UTC m=+6.129932965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:07.476700 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.476661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:07.477368 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.477340 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:07.477483 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.477376 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:07.477483 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.477405 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:07.477619 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.477497 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:09.477455308 +0000 UTC m=+6.231708563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:07.708834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.708658 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 22:06:06 +0000 UTC" deadline="2027-12-24 04:22:40.093223991 +0000 UTC" Jul 12 22:11:07.708834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.708725 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12702h11m32.384504128s" Jul 12 22:11:07.851080 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.851047 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:07.851263 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.851200 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:07.851691 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.851670 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:07.851815 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.851780 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:07.852021 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.851998 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:07.852154 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:07.852132 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:07.894951 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:07.894916 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" event={"ID":"81f992adf125e091bbf7fa1071ef9694","Type":"ContainerStarted","Data":"9cb993de0e82817330903210956d4118255b8051f48923922f349abb8e8dbc8d"} Jul 12 22:11:08.916369 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:08.916282 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" event={"ID":"0ad369be1433261009ab42f4c9aad4fc","Type":"ContainerStarted","Data":"1ef63e88657e078780eb19340a4a0be5400c778491e0f7fb6677e0b13584d260"} Jul 12 22:11:08.942571 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:08.941041 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-188.ec2.internal" podStartSLOduration=3.941018116 podStartE2EDuration="3.941018116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:07.91597109 +0000 UTC m=+4.670224351" watchObservedRunningTime="2026-07-12 22:11:08.941018116 +0000 UTC m=+5.695271377" Jul 12 22:11:09.392962 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.392871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:09.392962 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.392929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:09.393211 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.393078 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:09.393211 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.393145 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:13.393124518 +0000 UTC m=+10.147377758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:09.393669 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.393624 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:09.393761 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.393685 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:13.393667 +0000 UTC m=+10.147920238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:09.493819 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.493776 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:09.494034 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.494015 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:09.494123 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.494046 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:09.494123 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.494062 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:09.494236 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.494131 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:13.494108638 +0000 UTC m=+10.248361889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:09.855310 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.855278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:09.855503 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.855422 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:09.855825 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.855801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:09.855938 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.855921 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:09.856006 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.855969 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:09.856065 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:09.856048 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:09.925586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.925544 2579 generic.go:356] "Generic (PLEG): container finished" podID="0ad369be1433261009ab42f4c9aad4fc" containerID="1ef63e88657e078780eb19340a4a0be5400c778491e0f7fb6677e0b13584d260" exitCode=0 Jul 12 22:11:09.926052 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:09.925610 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" event={"ID":"0ad369be1433261009ab42f4c9aad4fc","Type":"ContainerDied","Data":"1ef63e88657e078780eb19340a4a0be5400c778491e0f7fb6677e0b13584d260"} Jul 12 22:11:11.851513 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:11.851446 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:11.851923 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:11.851610 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:11.852075 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:11.852056 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:11.852175 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:11.852155 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:11.852440 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:11.852420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:11.852569 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:11.852542 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:13.431762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:13.431194 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:13.431762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:13.431259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:13.431762 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.431386 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:13.431762 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.431441 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:21.431422617 +0000 UTC m=+18.185675852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:13.431762 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.431472 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:13.431762 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.431510 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:21.431498065 +0000 UTC m=+18.185751315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:13.532534 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:13.532495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:13.533185 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.532721 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:13.533185 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.532746 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:13.533185 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.532761 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:13.533185 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.532819 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:21.532799453 +0000 UTC m=+18.287052707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:13.851716 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:13.851627 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:13.851883 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.851730 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:13.851883 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:13.851763 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:13.851883 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:13.851800 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:13.852029 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.851894 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:13.852029 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:13.852016 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:15.855002 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:15.854976 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:15.855524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:15.854970 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:15.855524 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:15.855105 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:15.855524 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:15.855213 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:15.855524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:15.854988 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:15.855524 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:15.855336 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:17.854173 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:17.854140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:17.854648 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:17.854141 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:17.854648 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:17.854277 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:17.854648 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:17.854325 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:17.854648 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:17.854404 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:17.854648 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:17.854520 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:19.850978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:19.850945 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:19.850978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:19.850946 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:19.851501 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:19.851106 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:19.851501 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:19.851138 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:19.851501 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:19.851269 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:19.851501 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:19.851451 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:21.496593 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:21.496555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:21.496593 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:21.496596 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:21.497087 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.496705 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:21.497087 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.496742 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:21.497087 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.496763 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:37.4967457 +0000 UTC m=+34.250998944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:21.497087 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.496812 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:37.496792444 +0000 UTC m=+34.251045679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:21.597837 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:21.597804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:21.597989 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.597969 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:21.598025 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.597990 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:21.598025 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.598000 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:21.598082 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.598050 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:37.598036947 +0000 UTC m=+34.352290182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:21.850871 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:21.850788 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:21.851008 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:21.850801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:21.851008 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.850900 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:21.851008 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:21.850917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:21.851008 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.850981 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:21.851202 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:21.851073 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:23.851835 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:23.851799 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:23.852329 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:23.851887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:23.852329 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:23.851905 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:23.852329 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:23.852006 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:23.852329 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:23.852034 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:23.852329 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:23.852113 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:25.851424 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.851231 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:25.851871 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:25.851542 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:25.851871 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.851604 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:25.851871 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:25.851720 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:25.851871 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.851773 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:25.851871 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:25.851856 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:25.960347 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.960315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" event={"ID":"0ad369be1433261009ab42f4c9aad4fc","Type":"ContainerStarted","Data":"5a09be7085092eadbefefa5aa3434f485ab45e6432cbdb6a84b74f4d8f119d69"} Jul 12 22:11:25.961639 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.961617 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerStarted","Data":"8f66e6b12e23dc73f4bd09962430d35ff34cf9ade5694c9e79b8f69a8a54b98c"} Jul 12 22:11:25.962936 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.962907 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jtvzw" event={"ID":"6750b24d-b4dc-4842-b0a8-91d90c255345","Type":"ContainerStarted","Data":"f83e079a9b4be6117137ac43042ada91420315dd250bdeb2f13e21a1f75bff43"} Jul 12 22:11:25.964232 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.964210 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dd2q2" event={"ID":"165d9e3d-8328-4146-b17a-c3b9348a411c","Type":"ContainerStarted","Data":"1b4908750e430d05aa440e0996456276d7976e2b717ff881472edb6a315b4a19"} Jul 12 22:11:25.965487 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.965468 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"77cb7b7e7d82f69a8b7eb595b86c07be77e8d01db18fae6b7034c0784d6d43a7"} Jul 12 22:11:25.966753 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.966736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tbnjt" event={"ID":"7a5ffff7-47c3-4aad-8d8a-9be8675d178e","Type":"ContainerStarted","Data":"e6d105a786ef17ce52dd1e9443af2aa1975cdec0187c71c9b7cfd77ae1038df8"} Jul 12 22:11:25.968204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.968183 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-76dp8" event={"ID":"aba800a0-daee-4a0c-a746-f4bf1159460c","Type":"ContainerStarted","Data":"ac4af32d18292e39839ca83c67fb0e496af69439df003963a670e9fc034ecc84"} Jul 12 22:11:25.969641 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:25.969624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" event={"ID":"1f79d4b2-118f-4114-9543-04840dedd0c7","Type":"ContainerStarted","Data":"7d6a02b9e6c40bb3818ca1e126f9eb27e9e69b55b0c8e6b712fad99b90935955"} Jul 12 22:11:26.009086 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.009045 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jtvzw" podStartSLOduration=2.405488295 podStartE2EDuration="22.009029406s" totalImagesPullingTime="19.603541111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:26.008620812 +0000 UTC m=+22.762874070" watchObservedRunningTime="2026-07-12 22:11:26.009029406 +0000 UTC m=+22.763282663" Jul 12 22:11:26.023343 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.023296 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tbnjt" podStartSLOduration=13.437477939 podStartE2EDuration="22.023282907s" totalImagesPullingTime="8.585804968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:26.022880462 +0000 UTC m=+22.777133722" watchObservedRunningTime="2026-07-12 22:11:26.023282907 +0000 UTC m=+22.777536164" Jul 12 22:11:26.042031 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.041986 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-76dp8" podStartSLOduration=2.425906131 podStartE2EDuration="22.04196986s" totalImagesPullingTime="19.616063729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:26.041674659 +0000 UTC m=+22.795927925" watchObservedRunningTime="2026-07-12 22:11:26.04196986 +0000 UTC m=+22.796223116" Jul 12 22:11:26.058604 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.058548 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-188.ec2.internal" podStartSLOduration=21.05852729 podStartE2EDuration="21.05852729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:26.058495804 +0000 UTC m=+22.812749064" watchObservedRunningTime="2026-07-12 22:11:26.05852729 +0000 UTC m=+22.812780548" Jul 12 22:11:26.075489 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.075447 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dd2q2" podStartSLOduration=2.451121258 podStartE2EDuration="22.075432859s" totalImagesPullingTime="19.624311601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:26.075101003 +0000 UTC m=+22.829354260" watchObservedRunningTime="2026-07-12 22:11:26.075432859 +0000 UTC m=+22.829686116" Jul 12 22:11:26.098182 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.098154 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:26.351576 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.351264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:26.533214 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.533182 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:26.606177 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.606102 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:26.973823 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.973444 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rtcqm" event={"ID":"d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da","Type":"ContainerStarted","Data":"3c75274f1d2eb34cedcc0254569a9f4d0daf88c18a5b6270a728fd28e2d6beed"} Jul 12 22:11:26.978358 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978334 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:11:26.978676 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978656 2579 generic.go:356] "Generic (PLEG): container finished" podID="edbb9cc6-2861-494e-b593-f8a0cad404ea" containerID="f8eaa0572ed1ae761dd13684ab5afa38e9337d02c477a693a268f6b1c6dba1d7" exitCode=1 Jul 12 22:11:26.978753 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978735 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"d8bebf27251cfbc2e243d2554a275120e1ffbd455a305daf3ced14a6f6d45891"} Jul 12 22:11:26.978815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"bae3750aeecd8e65367edf3dbc0d37c9282795dce0e84930deb9739867583fed"} Jul 12 22:11:26.978852 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978840 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"70fc6251697a724da9f95193d910eab40a6e149dc8392d4d049a6c4ac8654cba"} Jul 12 22:11:26.978892 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978881 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"b2261648364a6154befb57a58232940bab1bef42346a3892274199d33b789b5f"} Jul 12 22:11:26.978937 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.978912 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerDied","Data":"f8eaa0572ed1ae761dd13684ab5afa38e9337d02c477a693a268f6b1c6dba1d7"} Jul 12 22:11:26.980228 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.980205 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cfdeee8-d3fc-4789-996f-0db8e40a4064" containerID="8f66e6b12e23dc73f4bd09962430d35ff34cf9ade5694c9e79b8f69a8a54b98c" exitCode=0 Jul 12 22:11:26.980334 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.980288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerDied","Data":"8f66e6b12e23dc73f4bd09962430d35ff34cf9ade5694c9e79b8f69a8a54b98c"} Jul 12 22:11:26.981551 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.981529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-scjjk" event={"ID":"45091bed-7fab-448b-94e6-1b98df48245b","Type":"ContainerStarted","Data":"b650db7d4c94dde14e07552c078416048e3387fa92c126ff031e51571a95cb59"} Jul 12 22:11:26.999136 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:26.999094 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rtcqm" podStartSLOduration=2.809361501 podStartE2EDuration="22.999081926s" totalImagesPullingTime="20.189720425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:26.99847677 +0000 UTC m=+23.752730027" watchObservedRunningTime="2026-07-12 22:11:26.999081926 +0000 UTC m=+23.753335182" Jul 12 22:11:27.044809 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.044764 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-scjjk" podStartSLOduration=3.501823558 podStartE2EDuration="23.044749061s" totalImagesPullingTime="19.542925503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:27.04422245 +0000 UTC m=+23.798475698" watchObservedRunningTime="2026-07-12 22:11:27.044749061 +0000 UTC m=+23.799002296" Jul 12 22:11:27.158300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.158257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:27.237092 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.237068 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 22:11:27.346915 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.346881 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:27.718282 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.718149 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T22:11:27.237085321Z","UUID":"e9d58e0c-fadc-44dd-81dc-09292f5e338e","Handler":null,"Name":"","Endpoint":""} Jul 12 22:11:27.719802 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.719650 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 22:11:27.719903 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.719820 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 22:11:27.750536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.750510 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:27.850628 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.850597 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:27.850808 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.850638 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:27.850808 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.850597 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:27.850808 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:27.850728 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:27.850808 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:27.850793 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:27.851023 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:27.850871 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:27.985570 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:27.985480 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" event={"ID":"1f79d4b2-118f-4114-9543-04840dedd0c7","Type":"ContainerStarted","Data":"58340423cdb2c8295cbf559babde484f4ab42ca1a8c05b45b35f604f4b41aeaa"} Jul 12 22:11:28.991969 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:28.991881 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:11:28.994562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:28.994529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"e39f2c29057984e6a5eabb757c149283320ad53b3ca6c0bbd6b23ab1fbe6151c"} Jul 12 22:11:28.997131 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:28.997107 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" event={"ID":"1f79d4b2-118f-4114-9543-04840dedd0c7","Type":"ContainerStarted","Data":"0f84ec9872a0bfaa268b4462bb6b1b5481c271bc512d698d4cce3d91531f9058"} Jul 12 22:11:29.018962 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:29.018910 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dg7mq" podStartSLOduration=6.437575612 podStartE2EDuration="25.018894151s" totalImagesPullingTime="18.581318539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:29.018319246 +0000 UTC m=+25.772572503" watchObservedRunningTime="2026-07-12 22:11:29.018894151 +0000 UTC m=+25.773147407" Jul 12 22:11:29.850980 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:29.850947 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:29.851164 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:29.850960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:29.851164 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:29.851078 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:29.851164 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:29.850960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:29.851164 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:29.851130 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:29.851380 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:29.851230 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:30.127937 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:30.127853 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:30.128658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:30.128520 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:31.001435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.001358 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:31.001976 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.001946 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dd2q2" Jul 12 22:11:31.694797 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.694635 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:31.795484 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.795451 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:31.851073 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.851042 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:31.851248 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.851077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:31.851248 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.851042 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:31.851248 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:31.851157 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:31.851248 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:31.851224 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:31.851397 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:31.851346 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:31.896253 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:31.896179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:32.005537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.005509 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:32.008819 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.008799 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:11:32.009271 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.009224 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"3477ac5ce0de3b77a347d7a5171ae99088bab0d9c6f82ad61a40281a037dfffa"} Jul 12 22:11:32.009620 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.009604 2579 scope.go:117] "RemoveContainer" containerID="f8eaa0572ed1ae761dd13684ab5afa38e9337d02c477a693a268f6b1c6dba1d7" Jul 12 22:11:32.184609 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.184583 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:32.279641 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.279616 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:32.786122 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.786094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:32.888089 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:32.888059 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.012824 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.012738 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cfdeee8-d3fc-4789-996f-0db8e40a4064" containerID="2153d28f00e2da0cb3a82e88105d1ddba43b888b253435276ff91edc9649c402" exitCode=0 Jul 12 22:11:33.013031 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.012823 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerDied","Data":"2153d28f00e2da0cb3a82e88105d1ddba43b888b253435276ff91edc9649c402"} Jul 12 22:11:33.016672 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.016653 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:11:33.017063 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.017042 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" event={"ID":"edbb9cc6-2861-494e-b593-f8a0cad404ea","Type":"ContainerStarted","Data":"502204922f64feaabdaeac15c614239bc8c46342a9c85a64970422c1083b446a"} Jul 12 22:11:33.017726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.017264 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:33.017726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.017337 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:33.017726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.017387 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:33.032936 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.032910 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:33.033099 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.033091 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:11:33.070420 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.070365 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" podStartSLOduration=9.204548865 podStartE2EDuration="29.070348013s" totalImagesPullingTime="19.865799148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:33.06968339 +0000 UTC m=+29.823936646" watchObservedRunningTime="2026-07-12 22:11:33.070348013 +0000 UTC m=+29.824601269" Jul 12 22:11:33.252255 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.252221 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.350833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.350796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.489287 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.489256 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.636453 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.636372 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.753784 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.753750 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.852556 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.852520 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:33.852974 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.852609 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:33.852974 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.852659 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:33.852974 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:33.852661 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:33.852974 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:33.852747 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:33.852974 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:33.852821 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:33.881463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.881431 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:33.983564 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:33.983478 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:34.789098 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.788842 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nj7st"] Jul 12 22:11:34.789268 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.789198 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:34.789351 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:34.789329 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:34.792388 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.792359 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lxs2"] Jul 12 22:11:34.792536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.792471 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:34.792599 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:34.792579 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:34.793010 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.792971 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7c5q2"] Jul 12 22:11:34.793111 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.793064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:34.793172 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:34.793153 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:34.991319 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:34.991212 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:35.024129 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:35.024093 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cfdeee8-d3fc-4789-996f-0db8e40a4064" containerID="8e922008cd003c55bfa801f6a6c217b2758dce6451284db443a4ea0d4a49b4bd" exitCode=0 Jul 12 22:11:35.024333 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:35.024169 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerDied","Data":"8e922008cd003c55bfa801f6a6c217b2758dce6451284db443a4ea0d4a49b4bd"} Jul 12 22:11:35.075926 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:35.075886 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:35.404118 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:35.404091 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:35.467432 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:35.467401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:36.851348 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:36.851303 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:36.851348 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:36.851344 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:36.851913 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:36.851312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:36.851913 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:36.851427 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:36.851913 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:36.851487 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:36.851913 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:36.851571 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:36.875275 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:36.875232 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:36.974361 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:36.974330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:37.031314 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.031281 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cfdeee8-d3fc-4789-996f-0db8e40a4064" containerID="4ae04883ad0ba202510b3b87116e5d39fc23a637afa422065be0588df08bcb30" exitCode=0 Jul 12 22:11:37.031463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.031333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerDied","Data":"4ae04883ad0ba202510b3b87116e5d39fc23a637afa422065be0588df08bcb30"} Jul 12 22:11:37.067049 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.067018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:37.140406 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.140326 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:37.525284 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.525252 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:37.525468 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.525297 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:37.525468 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.525384 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:37.525468 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.525448 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret podName:1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a nodeName:}" failed. No retries permitted until 2026-07-12 22:12:09.525432096 +0000 UTC m=+66.279685332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret") pod "global-pull-secret-syncer-nj7st" (UID: "1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:11:37.525603 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.525390 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:37.525603 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.525530 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs podName:58ae1d9a-b83c-4fa0-adb6-385298d8c5f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:12:09.525511184 +0000 UTC m=+66.279764424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs") pod "network-metrics-daemon-7c5q2" (UID: "58ae1d9a-b83c-4fa0-adb6-385298d8c5f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:11:37.626115 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:37.626086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:37.626309 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.626289 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:11:37.626363 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.626316 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:11:37.626363 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.626329 2579 projected.go:196] Error preparing data for projected volume kube-api-access-v8ngv for pod openshift-network-diagnostics/network-check-target-8lxs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:37.626452 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:37.626396 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv podName:9158ef47-9b2c-4638-ab6d-327ea3602716 nodeName:}" failed. No retries permitted until 2026-07-12 22:12:09.626374493 +0000 UTC m=+66.380627763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v8ngv" (UniqueName: "kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv") pod "network-check-target-8lxs2" (UID: "9158ef47-9b2c-4638-ab6d-327ea3602716") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:11:38.850979 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:38.850942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:38.850979 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:38.850967 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:38.851508 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:38.851072 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7c5q2" podUID="58ae1d9a-b83c-4fa0-adb6-385298d8c5f7" Jul 12 22:11:38.851508 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:38.851131 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nj7st" podUID="1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a" Jul 12 22:11:38.851508 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:38.851179 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:38.851508 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:38.851252 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lxs2" podUID="9158ef47-9b2c-4638-ab6d-327ea3602716" Jul 12 22:11:40.043355 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.043129 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-188.ec2.internal" event="NodeReady" Jul 12 22:11:40.043784 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.043468 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 22:11:40.103994 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.103956 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2ckzz"] Jul 12 22:11:40.106528 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.106505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.107642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.107613 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4f6cd"] Jul 12 22:11:40.109269 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.109233 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.109535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.109412 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 22:11:40.109726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.109703 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 22:11:40.109960 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.109944 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 22:11:40.110056 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.110026 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5sgr4\"" Jul 12 22:11:40.110119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.110062 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 22:11:40.112614 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.112594 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 22:11:40.112724 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.112610 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nwvjz\"" Jul 12 22:11:40.112724 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.112655 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 22:11:40.112838 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.112735 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 22:11:40.120447 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.120430 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2ckzz"] Jul 12 22:11:40.123174 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.123142 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4f6cd"] Jul 12 22:11:40.203338 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.203298 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5zrn4"] Jul 12 22:11:40.205305 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.205281 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.208385 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.208364 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 22:11:40.208546 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.208365 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r9hf\"" Jul 12 22:11:40.208637 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.208374 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 22:11:40.220204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.220179 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zrn4"] Jul 12 22:11:40.223951 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.223929 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f9b9c4c76-cwlpl"] Jul 12 22:11:40.225968 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.225948 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.228419 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.228398 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 22:11:40.228970 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.228550 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 22:11:40.228970 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.228724 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dcs2r\"" Jul 12 22:11:40.228970 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.228825 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 22:11:40.228970 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.228944 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 22:11:40.229208 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.229180 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 22:11:40.229607 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.229592 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 22:11:40.229684 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.229593 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 22:11:40.239972 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.239938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f9b9c4c76-cwlpl"] Jul 12 22:11:40.244407 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4b6e816-edc3-4854-b822-446e867f39c7-crio-socket\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.244548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlzqp\" (UniqueName: \"kubernetes.io/projected/a5769b66-5ecb-4684-95f3-bb079fea4583-kube-api-access-tlzqp\") pod \"ingress-canary-4f6cd\" (UID: \"a5769b66-5ecb-4684-95f3-bb079fea4583\") " pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.244548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4b6e816-edc3-4854-b822-446e867f39c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.244548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b6e816-edc3-4854-b822-446e867f39c7-data-volume\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.244548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5769b66-5ecb-4684-95f3-bb079fea4583-cert\") pod \"ingress-canary-4f6cd\" (UID: \"a5769b66-5ecb-4684-95f3-bb079fea4583\") " pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.244734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4b6e816-edc3-4854-b822-446e867f39c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.244734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.244662 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95jg\" (UniqueName: \"kubernetes.io/projected/d4b6e816-edc3-4854-b822-446e867f39c7-kube-api-access-n95jg\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.336269 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.336125 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:40.346024 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.345994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4b6e816-edc3-4854-b822-446e867f39c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346186 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95jg\" (UniqueName: \"kubernetes.io/projected/d4b6e816-edc3-4854-b822-446e867f39c7-kube-api-access-n95jg\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346186 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346175 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47713c46-5f1b-4b28-be04-9d7abbddb8ee-tmp-dir\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.346322 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47713c46-5f1b-4b28-be04-9d7abbddb8ee-config-volume\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.346322 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-config\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.346396 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4b6e816-edc3-4854-b822-446e867f39c7-crio-socket\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346396 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346371 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlzqp\" (UniqueName: \"kubernetes.io/projected/a5769b66-5ecb-4684-95f3-bb079fea4583-kube-api-access-tlzqp\") pod \"ingress-canary-4f6cd\" (UID: \"a5769b66-5ecb-4684-95f3-bb079fea4583\") " pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.346476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-oauth-serving-cert\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.346476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-serving-cert\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.346573 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-service-ca\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.346573 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2d8f\" (UniqueName: \"kubernetes.io/projected/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-kube-api-access-v2d8f\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.346573 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346548 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4b6e816-edc3-4854-b822-446e867f39c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346714 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346572 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtdcq\" (UniqueName: \"kubernetes.io/projected/47713c46-5f1b-4b28-be04-9d7abbddb8ee-kube-api-access-wtdcq\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.346714 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346602 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47713c46-5f1b-4b28-be04-9d7abbddb8ee-metrics-tls\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.346714 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346620 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4b6e816-edc3-4854-b822-446e867f39c7-crio-socket\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346714 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-oauth-config\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.346714 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346642 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4b6e816-edc3-4854-b822-446e867f39c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346933 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b6e816-edc3-4854-b822-446e867f39c7-data-volume\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.346933 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.346779 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5769b66-5ecb-4684-95f3-bb079fea4583-cert\") pod \"ingress-canary-4f6cd\" (UID: \"a5769b66-5ecb-4684-95f3-bb079fea4583\") " pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.347058 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.347033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b6e816-edc3-4854-b822-446e867f39c7-data-volume\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.351127 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.351102 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4b6e816-edc3-4854-b822-446e867f39c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.351286 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.351159 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5769b66-5ecb-4684-95f3-bb079fea4583-cert\") pod \"ingress-canary-4f6cd\" (UID: \"a5769b66-5ecb-4684-95f3-bb079fea4583\") " pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.354747 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.354723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95jg\" (UniqueName: \"kubernetes.io/projected/d4b6e816-edc3-4854-b822-446e867f39c7-kube-api-access-n95jg\") pod \"insights-runtime-extractor-2ckzz\" (UID: \"d4b6e816-edc3-4854-b822-446e867f39c7\") " pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.355039 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.355016 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlzqp\" (UniqueName: \"kubernetes.io/projected/a5769b66-5ecb-4684-95f3-bb079fea4583-kube-api-access-tlzqp\") pod \"ingress-canary-4f6cd\" (UID: \"a5769b66-5ecb-4684-95f3-bb079fea4583\") " pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.419781 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.419745 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2ckzz" Jul 12 22:11:40.427691 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.427666 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4f6cd" Jul 12 22:11:40.439759 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.439726 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:40.447862 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.447826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-serving-cert\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.447862 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.447873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-service-ca\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448081 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.447920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2d8f\" (UniqueName: \"kubernetes.io/projected/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-kube-api-access-v2d8f\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448081 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.447952 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtdcq\" (UniqueName: \"kubernetes.io/projected/47713c46-5f1b-4b28-be04-9d7abbddb8ee-kube-api-access-wtdcq\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.448081 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.447981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47713c46-5f1b-4b28-be04-9d7abbddb8ee-metrics-tls\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.448081 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448006 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-oauth-config\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448203 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47713c46-5f1b-4b28-be04-9d7abbddb8ee-tmp-dir\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.448203 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47713c46-5f1b-4b28-be04-9d7abbddb8ee-config-volume\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.448203 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448131 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-config\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448203 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-oauth-serving-cert\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448669 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47713c46-5f1b-4b28-be04-9d7abbddb8ee-tmp-dir\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.448946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448743 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-service-ca\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448785 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-oauth-serving-cert\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.448946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.448902 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47713c46-5f1b-4b28-be04-9d7abbddb8ee-config-volume\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.449252 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.449214 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-config\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.450810 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.450788 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-serving-cert\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.451090 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.451058 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-oauth-config\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.451174 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.451144 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47713c46-5f1b-4b28-be04-9d7abbddb8ee-metrics-tls\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.456888 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.456862 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtdcq\" (UniqueName: \"kubernetes.io/projected/47713c46-5f1b-4b28-be04-9d7abbddb8ee-kube-api-access-wtdcq\") pod \"dns-default-5zrn4\" (UID: \"47713c46-5f1b-4b28-be04-9d7abbddb8ee\") " pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.457438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.457414 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2d8f\" (UniqueName: \"kubernetes.io/projected/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-kube-api-access-v2d8f\") pod \"console-f9b9c4c76-cwlpl\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.515148 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.515108 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:40.536101 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.536069 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:11:40.850570 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.850535 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:11:40.850765 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.850535 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:11:40.850826 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.850536 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:11:40.853538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.853493 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 22:11:40.853538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.853522 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 22:11:40.853728 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.853588 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npkdh\"" Jul 12 22:11:40.853888 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.853870 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 22:11:40.853999 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.853945 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bgjrf\"" Jul 12 22:11:40.854464 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:40.854448 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 22:11:41.463805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.463776 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zrn4"] Jul 12 22:11:41.465053 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.465027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4f6cd"] Jul 12 22:11:41.467209 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.467181 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2ckzz"] Jul 12 22:11:41.468555 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:41.468526 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47713c46_5f1b_4b28_be04_9d7abbddb8ee.slice/crio-d543b8424c79cfeae1233454780daefa3edff95b347feaa117c960726389e534 WatchSource:0}: Error finding container d543b8424c79cfeae1233454780daefa3edff95b347feaa117c960726389e534: Status 404 returned error can't find the container with id d543b8424c79cfeae1233454780daefa3edff95b347feaa117c960726389e534 Jul 12 22:11:41.468993 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:41.468973 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5769b66_5ecb_4684_95f3_bb079fea4583.slice/crio-9b2703321fe22928c28cd9fdcec560ac01d806b0f2a473aefc83e2954daafa70 WatchSource:0}: Error finding container 9b2703321fe22928c28cd9fdcec560ac01d806b0f2a473aefc83e2954daafa70: Status 404 returned error can't find the container with id 9b2703321fe22928c28cd9fdcec560ac01d806b0f2a473aefc83e2954daafa70 Jul 12 22:11:41.476196 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.476127 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f9b9c4c76-cwlpl"] Jul 12 22:11:41.480793 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:41.480559 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b6e816_edc3_4854_b822_446e867f39c7.slice/crio-a5cea7e7a54be2d2b3efc06bb8dcf7b5cdf0376fdfc7c6606d11359be4ad9f05 WatchSource:0}: Error finding container a5cea7e7a54be2d2b3efc06bb8dcf7b5cdf0376fdfc7c6606d11359be4ad9f05: Status 404 returned error can't find the container with id a5cea7e7a54be2d2b3efc06bb8dcf7b5cdf0376fdfc7c6606d11359be4ad9f05 Jul 12 22:11:41.482079 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:41.482060 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e36c7f_e5cd_466c_9e58_109cd1cf9bdf.slice/crio-5d81164d65ec689f97e5d62e7bf25b46ff6cfd0418e938bd7b3132d152717f40 WatchSource:0}: Error finding container 5d81164d65ec689f97e5d62e7bf25b46ff6cfd0418e938bd7b3132d152717f40: Status 404 returned error can't find the container with id 5d81164d65ec689f97e5d62e7bf25b46ff6cfd0418e938bd7b3132d152717f40 Jul 12 22:11:41.743478 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.743447 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:41.820275 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.820158 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:41.901422 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.901383 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:41.908447 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.908421 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs"] Jul 12 22:11:41.910305 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.910258 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:41.912667 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.912647 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 12 22:11:41.912779 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.912764 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-v95dg\"" Jul 12 22:11:41.922230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.922211 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs"] Jul 12 22:11:41.959591 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:41.959551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54d14e9-eb17-4ff8-837e-ce6b8580b44a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-q5vgs\" (UID: \"e54d14e9-eb17-4ff8-837e-ce6b8580b44a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:42.046016 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.045926 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9b9c4c76-cwlpl" event={"ID":"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf","Type":"ContainerStarted","Data":"5d81164d65ec689f97e5d62e7bf25b46ff6cfd0418e938bd7b3132d152717f40"} Jul 12 22:11:42.047579 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.047504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zrn4" event={"ID":"47713c46-5f1b-4b28-be04-9d7abbddb8ee","Type":"ContainerStarted","Data":"d543b8424c79cfeae1233454780daefa3edff95b347feaa117c960726389e534"} Jul 12 22:11:42.052627 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.052584 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cfdeee8-d3fc-4789-996f-0db8e40a4064" containerID="8105f030cd52b8dfaec3ba3a0c69f648e78a899d76018a7283c2f4b2e5161374" exitCode=0 Jul 12 22:11:42.052752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.052670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerDied","Data":"8105f030cd52b8dfaec3ba3a0c69f648e78a899d76018a7283c2f4b2e5161374"} Jul 12 22:11:42.054058 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.054033 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4f6cd" event={"ID":"a5769b66-5ecb-4684-95f3-bb079fea4583","Type":"ContainerStarted","Data":"9b2703321fe22928c28cd9fdcec560ac01d806b0f2a473aefc83e2954daafa70"} Jul 12 22:11:42.056325 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.056256 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ckzz" event={"ID":"d4b6e816-edc3-4854-b822-446e867f39c7","Type":"ContainerStarted","Data":"a5cea7e7a54be2d2b3efc06bb8dcf7b5cdf0376fdfc7c6606d11359be4ad9f05"} Jul 12 22:11:42.060261 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.060217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54d14e9-eb17-4ff8-837e-ce6b8580b44a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-q5vgs\" (UID: \"e54d14e9-eb17-4ff8-837e-ce6b8580b44a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:42.060667 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:42.060586 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 12 22:11:42.060667 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:42.060651 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54d14e9-eb17-4ff8-837e-ce6b8580b44a-tls-certificates podName:e54d14e9-eb17-4ff8-837e-ce6b8580b44a nodeName:}" failed. No retries permitted until 2026-07-12 22:11:42.560631966 +0000 UTC m=+39.314885202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/e54d14e9-eb17-4ff8-837e-ce6b8580b44a-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-q5vgs" (UID: "e54d14e9-eb17-4ff8-837e-ce6b8580b44a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 12 22:11:42.145303 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.145206 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:42.564293 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.564249 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54d14e9-eb17-4ff8-837e-ce6b8580b44a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-q5vgs\" (UID: \"e54d14e9-eb17-4ff8-837e-ce6b8580b44a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:42.580300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.580165 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54d14e9-eb17-4ff8-837e-ce6b8580b44a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-q5vgs\" (UID: \"e54d14e9-eb17-4ff8-837e-ce6b8580b44a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:42.817925 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.817854 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:42.819517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.819284 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:42.887632 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:42.887547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:43.004950 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:43.004813 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs"] Jul 12 22:11:43.066612 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:43.066577 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cfdeee8-d3fc-4789-996f-0db8e40a4064" containerID="ff57b235ddbb73e2f1ee8e07f25e56146725abfef6aaac10f2f67a11e06be2cd" exitCode=0 Jul 12 22:11:43.066792 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:43.066698 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerDied","Data":"ff57b235ddbb73e2f1ee8e07f25e56146725abfef6aaac10f2f67a11e06be2cd"} Jul 12 22:11:43.073429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:43.073373 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" event={"ID":"e54d14e9-eb17-4ff8-837e-ce6b8580b44a","Type":"ContainerStarted","Data":"844406adb336d6673f0d7b584e554865c78d4942683cb655a383a867e8fdb890"} Jul 12 22:11:44.081970 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:44.081673 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" event={"ID":"1cfdeee8-d3fc-4789-996f-0db8e40a4064","Type":"ContainerStarted","Data":"b00d93c4b5275e8b263e819eca5ab32fa60a12af3499b7fa49fb6a4f1c961254"} Jul 12 22:11:44.108466 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:44.108410 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2fbdv" podStartSLOduration=9.067280384 podStartE2EDuration="40.108389384s" totalImagesPullingTime="31.041109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:44.107071967 +0000 UTC m=+40.861325225" watchObservedRunningTime="2026-07-12 22:11:44.108389384 +0000 UTC m=+40.862642643" Jul 12 22:11:44.123990 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:44.123953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:44.234768 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:44.234737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:44.306399 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:44.306364 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:47.786125 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:47.786089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:47.854780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:47.854701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:48.291340 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:48.291250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:48.357630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:48.357593 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:49.395094 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.395063 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59bc9c9589-q2clp"] Jul 12 22:11:49.426365 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.426334 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59bc9c9589-q2clp"] Jul 12 22:11:49.426546 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.426438 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.436146 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.436121 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 22:11:49.518140 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-service-ca\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.518356 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-oauth-serving-cert\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.518356 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-serving-cert\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.518356 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-console-config\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.518356 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518295 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-oauth-config\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.518356 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518334 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/43c0831f-0175-43f3-8744-fa70e923f318-kube-api-access-cd9l6\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.518579 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.518366 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-trusted-ca-bundle\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619111 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619075 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-service-ca\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619111 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-oauth-serving-cert\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619372 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619268 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-serving-cert\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619372 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-console-config\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619372 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-oauth-config\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619371 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/43c0831f-0175-43f3-8744-fa70e923f318-kube-api-access-cd9l6\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-trusted-ca-bundle\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619868 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619844 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-oauth-serving-cert\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.619975 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619940 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-service-ca\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.620025 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.619978 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-console-config\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.620317 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.620295 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-trusted-ca-bundle\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.622527 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.622501 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-serving-cert\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.622612 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.622500 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-oauth-config\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.628230 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.628210 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/43c0831f-0175-43f3-8744-fa70e923f318-kube-api-access-cd9l6\") pod \"console-59bc9c9589-q2clp\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.737189 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.737148 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:49.981444 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:49.981361 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:50.081520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:50.081491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:50.209677 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:50.209561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:50.220626 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:50.220599 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59bc9c9589-q2clp"] Jul 12 22:11:50.224368 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:50.224326 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c0831f_0175_43f3_8744_fa70e923f318.slice/crio-4f7b3fb7449fd47abe16c18da9c4319e272220c87c20fb7e157a88cf0584de31 WatchSource:0}: Error finding container 4f7b3fb7449fd47abe16c18da9c4319e272220c87c20fb7e157a88cf0584de31: Status 404 returned error can't find the container with id 4f7b3fb7449fd47abe16c18da9c4319e272220c87c20fb7e157a88cf0584de31 Jul 12 22:11:50.307861 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:50.307689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:51.103034 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.102987 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9b9c4c76-cwlpl" event={"ID":"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf","Type":"ContainerStarted","Data":"fb822ebaa2e89e58484aedc11bdbe534b02347b997b58b68b26bc9635d334b1d"} Jul 12 22:11:51.104712 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.104689 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zrn4" event={"ID":"47713c46-5f1b-4b28-be04-9d7abbddb8ee","Type":"ContainerStarted","Data":"b10fd84934288cb2eaf843f0c015161eb3f9ebc59e2fc414b2473c5ee8b9a81e"} Jul 12 22:11:51.104826 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.104730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zrn4" event={"ID":"47713c46-5f1b-4b28-be04-9d7abbddb8ee","Type":"ContainerStarted","Data":"8b98ea0a3a329f62d6622702d0ea462b70fe2d0c85c9c1d779bfea65f2c62600"} Jul 12 22:11:51.104826 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.104755 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5zrn4" Jul 12 22:11:51.106012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.105991 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4f6cd" event={"ID":"a5769b66-5ecb-4684-95f3-bb079fea4583","Type":"ContainerStarted","Data":"1ba782ebaed59da6eaa96e416c95cc7a0f625255a6bb5962959ce26386ffeb25"} Jul 12 22:11:51.107383 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.107358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" event={"ID":"e54d14e9-eb17-4ff8-837e-ce6b8580b44a","Type":"ContainerStarted","Data":"7e93ec46c5f1198075ab852c3956735a5ae6a563029cf019ee3e60be7f1496ce"} Jul 12 22:11:51.107547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.107534 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:51.108849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.108827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bc9c9589-q2clp" event={"ID":"43c0831f-0175-43f3-8744-fa70e923f318","Type":"ContainerStarted","Data":"e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18"} Jul 12 22:11:51.108943 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.108861 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bc9c9589-q2clp" event={"ID":"43c0831f-0175-43f3-8744-fa70e923f318","Type":"ContainerStarted","Data":"4f7b3fb7449fd47abe16c18da9c4319e272220c87c20fb7e157a88cf0584de31"} Jul 12 22:11:51.110269 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.110227 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ckzz" event={"ID":"d4b6e816-edc3-4854-b822-446e867f39c7","Type":"ContainerStarted","Data":"d269b31efd198ed832cf37a54f224f28cac87a4cbb88dd421cb23cbda27275e3"} Jul 12 22:11:51.112762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.112742 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" Jul 12 22:11:51.124017 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.123963 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f9b9c4c76-cwlpl" podStartSLOduration=2.5207642310000002 podStartE2EDuration="11.123944935s" totalImagesPullingTime="8.603180704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:51.123162301 +0000 UTC m=+47.877415559" watchObservedRunningTime="2026-07-12 22:11:51.123944935 +0000 UTC m=+47.878198173" Jul 12 22:11:51.141874 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.141833 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5zrn4" podStartSLOduration=2.95706881 podStartE2EDuration="11.141817196s" totalImagesPullingTime="8.184748386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:51.141402424 +0000 UTC m=+47.895655680" watchObservedRunningTime="2026-07-12 22:11:51.141817196 +0000 UTC m=+47.896070453" Jul 12 22:11:51.164683 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.164633 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4f6cd" podStartSLOduration=2.980614854 podStartE2EDuration="11.164619676s" totalImagesPullingTime="8.184004822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:51.164437892 +0000 UTC m=+47.918691152" watchObservedRunningTime="2026-07-12 22:11:51.164619676 +0000 UTC m=+47.918872912" Jul 12 22:11:51.183880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.183831 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-q5vgs" podStartSLOduration=3.111336328 podStartE2EDuration="10.183816026s" totalImagesPullingTime="7.072479698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:51.183290786 +0000 UTC m=+47.937544042" watchObservedRunningTime="2026-07-12 22:11:51.183816026 +0000 UTC m=+47.938069283" Jul 12 22:11:51.208688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.208639 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59bc9c9589-q2clp" podStartSLOduration=2.208617828 podStartE2EDuration="2.208617828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:51.208133771 +0000 UTC m=+47.962387029" watchObservedRunningTime="2026-07-12 22:11:51.208617828 +0000 UTC m=+47.962871084" Jul 12 22:11:51.940628 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.940590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:51.993752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:51.993717 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lxt58"] Jul 12 22:11:52.009921 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.009889 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.016347 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.016315 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 12 22:11:52.016506 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.016374 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 22:11:52.016506 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.016485 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 22:11:52.022637 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.022610 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 12 22:11:52.027775 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.022611 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hhrtt\"" Jul 12 22:11:52.027775 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.023705 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 22:11:52.027775 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.024452 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lxt58"] Jul 12 22:11:52.038681 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.038659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.038789 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.038694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.038789 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.038715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.038789 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.038772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrkqn\" (UniqueName: \"kubernetes.io/projected/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-kube-api-access-vrkqn\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.044519 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.044480 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:52.115788 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.115748 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ckzz" event={"ID":"d4b6e816-edc3-4854-b822-446e867f39c7","Type":"ContainerStarted","Data":"a6f9c04c51391b4a23c5a36da3d62b7f71b7d475ef2052ba0279c05e4487954f"} Jul 12 22:11:52.140498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.139465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.140498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.139512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.140498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.139546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.140498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.139695 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrkqn\" (UniqueName: \"kubernetes.io/projected/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-kube-api-access-vrkqn\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.140778 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.140633 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.142826 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.142800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.142944 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.142852 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.145413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.145387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:52.150805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.150775 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrkqn\" (UniqueName: \"kubernetes.io/projected/db3db6c9-e7b1-4099-bbb9-797b0ef02b21-kube-api-access-vrkqn\") pod \"prometheus-operator-64785b6675-lxt58\" (UID: \"db3db6c9-e7b1-4099-bbb9-797b0ef02b21\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.336472 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.336437 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" Jul 12 22:11:52.525372 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.525339 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lxt58"] Jul 12 22:11:52.525851 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:52.525825 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3db6c9_e7b1_4099_bbb9_797b0ef02b21.slice/crio-1f72657aea137b83f20b0c0a90629378272ef345687cbf7212a9a2d740ef5e6b WatchSource:0}: Error finding container 1f72657aea137b83f20b0c0a90629378272ef345687cbf7212a9a2d740ef5e6b: Status 404 returned error can't find the container with id 1f72657aea137b83f20b0c0a90629378272ef345687cbf7212a9a2d740ef5e6b Jul 12 22:11:52.827482 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.827449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:52.924040 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:52.924008 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:53.121202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:53.121106 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" event={"ID":"db3db6c9-e7b1-4099-bbb9-797b0ef02b21","Type":"ContainerStarted","Data":"1f72657aea137b83f20b0c0a90629378272ef345687cbf7212a9a2d740ef5e6b"} Jul 12 22:11:53.201823 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:53.201787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:53.304657 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:53.304617 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:53.999579 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:53.999537 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:54.101546 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:54.101508 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:54.817672 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:54.817634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:54.917945 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:54.917912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:55.055455 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:55.055419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:55.129093 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:55.129056 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ckzz" event={"ID":"d4b6e816-edc3-4854-b822-446e867f39c7","Type":"ContainerStarted","Data":"398b6d1f282512b85ef642306b79f6647a64d6e5e38d14e477adff5b485191ec"} Jul 12 22:11:55.130750 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:55.130716 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" event={"ID":"db3db6c9-e7b1-4099-bbb9-797b0ef02b21","Type":"ContainerStarted","Data":"cfa941f4b5ee7881bb616a08917c851d59bdc8a3051c9ee3c449ec8463c6fdb9"} Jul 12 22:11:55.130876 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:55.130769 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" event={"ID":"db3db6c9-e7b1-4099-bbb9-797b0ef02b21","Type":"ContainerStarted","Data":"f8152a0a4d0443eb086e5c47af128fff8c9cce4f005d446e71bb7e4c57805fe7"} Jul 12 22:11:55.147971 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:55.147923 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2ckzz" podStartSLOduration=2.248783899 podStartE2EDuration="15.147907183s" totalImagesPullingTime="12.899123284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:55.146526862 +0000 UTC m=+51.900780118" watchObservedRunningTime="2026-07-12 22:11:55.147907183 +0000 UTC m=+51.902160417" Jul 12 22:11:55.164574 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:55.164518 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-lxt58" podStartSLOduration=1.9616813579999999 podStartE2EDuration="4.164503473s" totalImagesPullingTime="2.202822115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:55.163592265 +0000 UTC m=+51.917845526" watchObservedRunningTime="2026-07-12 22:11:55.164503473 +0000 UTC m=+51.918756730" Jul 12 22:11:56.904561 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:56.904528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.004510 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.004473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.134623 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.134591 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.568899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.568856 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.647428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.647393 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn"] Jul 12 22:11:57.669091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.669024 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn"] Jul 12 22:11:57.669291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.669187 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.669955 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.669933 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-slkwk"] Jul 12 22:11:57.672866 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.672841 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 12 22:11:57.673065 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.673049 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 12 22:11:57.673160 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.673130 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fgzg7\"" Jul 12 22:11:57.683451 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.683433 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.693102 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.693062 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 12 22:11:57.693296 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.693282 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 12 22:11:57.693513 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.693500 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-chk8m\"" Jul 12 22:11:57.693585 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.693572 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 12 22:11:57.696272 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.696234 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-slkwk"] Jul 12 22:11:57.715793 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.715770 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h8dht"] Jul 12 22:11:57.717497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.717477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.737810 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.737782 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.741660 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.741644 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 22:11:57.742996 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.742974 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 22:11:57.743183 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.743112 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k7pht\"" Jul 12 22:11:57.743472 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.743456 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 22:11:57.788818 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.788785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-wtmp\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.788818 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.788824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.789047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.788841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-sys\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.788870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.789047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.788926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-textfile\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-root\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.789278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjkpp\" (UniqueName: \"kubernetes.io/projected/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-api-access-hjkpp\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.789278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789115 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.789278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.789278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789212 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x66x\" (UniqueName: \"kubernetes.io/projected/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-kube-api-access-7x66x\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789284 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789312 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-tls\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789366 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-metrics-client-ca\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc9v2\" (UniqueName: \"kubernetes.io/projected/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-kube-api-access-bc9v2\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.789547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.789444 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.822944 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.822871 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.890728 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.890728 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x66x\" (UniqueName: \"kubernetes.io/projected/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-kube-api-access-7x66x\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-tls\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890853 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-metrics-client-ca\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc9v2\" (UniqueName: \"kubernetes.io/projected/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-kube-api-access-bc9v2\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.890966 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.890931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.891396 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:57.891007 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 12 22:11:57.891396 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:11:57.891073 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-tls podName:f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:58.39105163 +0000 UTC m=+55.145304885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-jx2rn" (UID: "f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95") : secret "openshift-state-metrics-tls" not found Jul 12 22:11:57.891497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.891497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-wtmp\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891481 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.891497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891479 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-metrics-client-ca\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-sys\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891479 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-textfile\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891611 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-wtmp\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-sys\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891637 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-root\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891684 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.891710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjkpp\" (UniqueName: \"kubernetes.io/projected/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-api-access-hjkpp\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.892188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.892188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891749 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-root\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.892188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.891770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.892435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.892408 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.892598 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.892445 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.892698 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.892654 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-textfile\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.893784 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.893762 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.894051 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.894033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-node-exporter-tls\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.894253 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.894217 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.894302 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.894271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.894355 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.894339 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.908375 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.908349 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjkpp\" (UniqueName: \"kubernetes.io/projected/708dc70e-0fea-42bd-8ec9-1ae79b77f4fb-kube-api-access-hjkpp\") pod \"kube-state-metrics-785778ff74-slkwk\" (UID: \"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:57.911354 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.911335 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x66x\" (UniqueName: \"kubernetes.io/projected/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-kube-api-access-7x66x\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:57.916549 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.916531 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc9v2\" (UniqueName: \"kubernetes.io/projected/c10680b7-41b7-4323-90cd-3e69a3f9b1c8-kube-api-access-bc9v2\") pod \"node-exporter-h8dht\" (UID: \"c10680b7-41b7-4323-90cd-3e69a3f9b1c8\") " pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:57.935867 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.935841 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:57.991840 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:57.991806 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" Jul 12 22:11:58.036599 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.036572 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:58.047669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.047631 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h8dht" Jul 12 22:11:58.078351 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:58.078318 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10680b7_41b7_4323_90cd_3e69a3f9b1c8.slice/crio-2fd7ce4c2b9c6ce6311a19e28fcf1356977f7d4b2283e3cbc8575c198e237b79 WatchSource:0}: Error finding container 2fd7ce4c2b9c6ce6311a19e28fcf1356977f7d4b2283e3cbc8575c198e237b79: Status 404 returned error can't find the container with id 2fd7ce4c2b9c6ce6311a19e28fcf1356977f7d4b2283e3cbc8575c198e237b79 Jul 12 22:11:58.140707 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.140665 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dht" event={"ID":"c10680b7-41b7-4323-90cd-3e69a3f9b1c8","Type":"ContainerStarted","Data":"2fd7ce4c2b9c6ce6311a19e28fcf1356977f7d4b2283e3cbc8575c198e237b79"} Jul 12 22:11:58.142999 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.142974 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:58.157112 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.157089 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-slkwk"] Jul 12 22:11:58.159422 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:58.159393 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708dc70e_0fea_42bd_8ec9_1ae79b77f4fb.slice/crio-88ec37948684789d3bb509f9756f1a413ea58c9f3c49740daac882741ad0b3be WatchSource:0}: Error finding container 88ec37948684789d3bb509f9756f1a413ea58c9f3c49740daac882741ad0b3be: Status 404 returned error can't find the container with id 88ec37948684789d3bb509f9756f1a413ea58c9f3c49740daac882741ad0b3be Jul 12 22:11:58.244674 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.244644 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:58.357304 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.357253 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:58.395723 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.395690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:58.397956 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.397927 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-jx2rn\" (UID: \"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:58.461393 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.461361 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:58.470391 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.470347 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 22:11:58.473535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.473515 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.477203 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477183 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 12 22:11:58.477324 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477203 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 12 22:11:58.477324 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 12 22:11:58.477324 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477206 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 12 22:11:58.477324 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 12 22:11:58.477677 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477663 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 12 22:11:58.477875 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477862 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 12 22:11:58.477933 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477898 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 12 22:11:58.477984 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.477898 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gtd9h\"" Jul 12 22:11:58.487990 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.487972 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 12 22:11:58.497767 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.497750 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 22:11:58.578839 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.578803 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" Jul 12 22:11:58.597310 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcg9r\" (UniqueName: \"kubernetes.io/projected/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-kube-api-access-bcg9r\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-web-config\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597431 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-config-volume\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-config-out\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597700 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597837 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597759 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597837 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597837 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.597955 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.597846 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.671726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.671689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:58.698909 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.698874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699080 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.698926 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699080 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.698960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699080 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.698990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699080 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699080 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcg9r\" (UniqueName: \"kubernetes.io/projected/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-kube-api-access-bcg9r\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-web-config\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-config-volume\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699257 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699301 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-config-out\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.699969 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.700061 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.699984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.701025 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.700996 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.702725 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.702587 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.702883 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.702861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.703343 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.703303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-config-out\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.703696 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.703667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.705022 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.704994 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-web-config\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.705173 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.705156 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.705255 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.705177 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.705616 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.705581 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-config-volume\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.707214 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.707193 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.718847 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.718818 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcg9r\" (UniqueName: \"kubernetes.io/projected/bbc1e1c4-0292-48b5-bc12-3d61242d4f15-kube-api-access-bcg9r\") pod \"alertmanager-main-0\" (UID: \"bbc1e1c4-0292-48b5-bc12-3d61242d4f15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:58.727612 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.727584 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn"] Jul 12 22:11:58.730645 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:58.730617 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ed0ad5_c3e4_4c61_a59f_7bd0d4a3ca95.slice/crio-285b536b8f2c914287c19aea091093ab4b9be4999ad24d24db74dfd1939bf861 WatchSource:0}: Error finding container 285b536b8f2c914287c19aea091093ab4b9be4999ad24d24db74dfd1939bf861: Status 404 returned error can't find the container with id 285b536b8f2c914287c19aea091093ab4b9be4999ad24d24db74dfd1939bf861 Jul 12 22:11:58.783457 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:58.783414 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 22:11:59.151133 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.151088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dht" event={"ID":"c10680b7-41b7-4323-90cd-3e69a3f9b1c8","Type":"ContainerStarted","Data":"a4c6401ffb3ec10252badcb3964a3a76dec841123eee8e7ab79f3578150fa441"} Jul 12 22:11:59.155209 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.155177 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" event={"ID":"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95","Type":"ContainerStarted","Data":"225e348158c54bdb0b5310b3e891d0b65bd313a00a027ec2e2197f61cbc12643"} Jul 12 22:11:59.155363 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.155258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" event={"ID":"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95","Type":"ContainerStarted","Data":"a778678bcb7c98e06f24ebd3dcf425cf4508c340f0b6a768c73b9f969daa8e13"} Jul 12 22:11:59.155363 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.155283 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" event={"ID":"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95","Type":"ContainerStarted","Data":"285b536b8f2c914287c19aea091093ab4b9be4999ad24d24db74dfd1939bf861"} Jul 12 22:11:59.157635 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.157573 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" event={"ID":"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb","Type":"ContainerStarted","Data":"88ec37948684789d3bb509f9756f1a413ea58c9f3c49740daac882741ad0b3be"} Jul 12 22:11:59.182057 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.181965 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 22:11:59.470491 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:11:59.470426 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc1e1c4_0292_48b5_bc12_3d61242d4f15.slice/crio-98a2a19468132efd9423c8e271b49fb0c34b52c8f8f23048338a63b074fdc441 WatchSource:0}: Error finding container 98a2a19468132efd9423c8e271b49fb0c34b52c8f8f23048338a63b074fdc441: Status 404 returned error can't find the container with id 98a2a19468132efd9423c8e271b49fb0c34b52c8f8f23048338a63b074fdc441 Jul 12 22:11:59.737999 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.737916 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:59.737999 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.737987 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:11:59.739204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.739172 2579 patch_prober.go:28] interesting pod/console-59bc9c9589-q2clp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 12 22:11:59.739328 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:11:59.739229 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-59bc9c9589-q2clp" podUID="43c0831f-0175-43f3-8744-fa70e923f318" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 12 22:12:00.164229 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.164193 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" event={"ID":"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb","Type":"ContainerStarted","Data":"85f041a24180489aaa28a978da1d224fd9d8ce1bd27070637b12013338cf5fc0"} Jul 12 22:12:00.164642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.164282 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" event={"ID":"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb","Type":"ContainerStarted","Data":"bf50327c6cc82625ab0adfbec33915227a3efea595d86952ff88478a8eb473e6"} Jul 12 22:12:00.164642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.164315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" event={"ID":"708dc70e-0fea-42bd-8ec9-1ae79b77f4fb","Type":"ContainerStarted","Data":"4c5321df2dced86f359a6cb6003a637d9f5f7d4f9c5f2b1f799be646f9106219"} Jul 12 22:12:00.165969 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.165933 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"98a2a19468132efd9423c8e271b49fb0c34b52c8f8f23048338a63b074fdc441"} Jul 12 22:12:00.167431 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.167404 2579 generic.go:356] "Generic (PLEG): container finished" podID="c10680b7-41b7-4323-90cd-3e69a3f9b1c8" containerID="a4c6401ffb3ec10252badcb3964a3a76dec841123eee8e7ab79f3578150fa441" exitCode=0 Jul 12 22:12:00.167550 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.167436 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dht" event={"ID":"c10680b7-41b7-4323-90cd-3e69a3f9b1c8","Type":"ContainerDied","Data":"a4c6401ffb3ec10252badcb3964a3a76dec841123eee8e7ab79f3578150fa441"} Jul 12 22:12:00.191857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.191813 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-slkwk" podStartSLOduration=1.858525957 podStartE2EDuration="3.191792339s" totalImagesPullingTime="1.333266382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:00.187778834 +0000 UTC m=+56.942032092" watchObservedRunningTime="2026-07-12 22:12:00.191792339 +0000 UTC m=+56.946045596" Jul 12 22:12:00.380583 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.380526 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:00.484162 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.484034 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:00.495748 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.495715 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz"] Jul 12 22:12:00.498757 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.498733 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.501540 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501512 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ml72n\"" Jul 12 22:12:00.501663 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501609 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 12 22:12:00.501663 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501609 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 12 22:12:00.501781 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501662 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 12 22:12:00.501781 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501609 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 12 22:12:00.501781 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501744 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c0eoupp3v54rh\"" Jul 12 22:12:00.501967 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.501950 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 12 22:12:00.517664 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.517640 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz"] Jul 12 22:12:00.536754 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.536721 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:12:00.536895 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.536793 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:12:00.538520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.538401 2579 patch_prober.go:28] interesting pod/console-f9b9c4c76-cwlpl container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 12 22:12:00.538520 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.538453 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-f9b9c4c76-cwlpl" podUID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 12 22:12:00.584341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.584234 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:00.618347 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618480 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618410 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618592 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618570 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m79br\" (UniqueName: \"kubernetes.io/projected/7eae20f3-801a-4c00-837a-b3b62f6f1461-kube-api-access-m79br\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618637 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eae20f3-801a-4c00-837a-b3b62f6f1461-metrics-client-ca\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618743 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-grpc-tls\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618799 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618764 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-tls\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.618897 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.618864 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.719892 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.719851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720079 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.719920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-tls\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720079 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.719947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720079 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.719996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720079 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.720059 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720330 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.720105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m79br\" (UniqueName: \"kubernetes.io/projected/7eae20f3-801a-4c00-837a-b3b62f6f1461-kube-api-access-m79br\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720330 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.720135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eae20f3-801a-4c00-837a-b3b62f6f1461-metrics-client-ca\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720330 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.720192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-grpc-tls\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.720992 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.720960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eae20f3-801a-4c00-837a-b3b62f6f1461-metrics-client-ca\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.722867 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.722839 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-tls\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.723002 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.722969 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.723124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.723050 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.723124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.723066 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.723214 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.723199 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-grpc-tls\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.723401 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.723384 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eae20f3-801a-4c00-837a-b3b62f6f1461-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.732579 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.732557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m79br\" (UniqueName: \"kubernetes.io/projected/7eae20f3-801a-4c00-837a-b3b62f6f1461-kube-api-access-m79br\") pod \"thanos-querier-56b4ddd8b5-npbnz\" (UID: \"7eae20f3-801a-4c00-837a-b3b62f6f1461\") " pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.808935 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.808893 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:00.950012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:00.949963 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz"] Jul 12 22:12:00.951917 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:00.951884 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eae20f3_801a_4c00_837a_b3b62f6f1461.slice/crio-0fd8e56312f43688441886be1ed7b71875af3a206ed1d70a6480638aa4a69fcf WatchSource:0}: Error finding container 0fd8e56312f43688441886be1ed7b71875af3a206ed1d70a6480638aa4a69fcf: Status 404 returned error can't find the container with id 0fd8e56312f43688441886be1ed7b71875af3a206ed1d70a6480638aa4a69fcf Jul 12 22:12:01.103003 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.102921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:01.119223 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.119195 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5zrn4" Jul 12 22:12:01.173669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.173636 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dht" event={"ID":"c10680b7-41b7-4323-90cd-3e69a3f9b1c8","Type":"ContainerStarted","Data":"8619f1e530aa25332e19a5a75aba55a2c39c36b091c688007da8f38837f5b5ee"} Jul 12 22:12:01.174104 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.173711 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dht" event={"ID":"c10680b7-41b7-4323-90cd-3e69a3f9b1c8","Type":"ContainerStarted","Data":"e0ef2f480621a372a6656f810bbdfcccc4c6e2eb746bead21f0b570219dae037"} Jul 12 22:12:01.175813 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.175788 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" event={"ID":"f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95","Type":"ContainerStarted","Data":"5852f61e40c363701bada2b7d3aabceeb50c1d7224250c017f21e95bd397a084"} Jul 12 22:12:01.177434 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.177406 2579 generic.go:356] "Generic (PLEG): container finished" podID="bbc1e1c4-0292-48b5-bc12-3d61242d4f15" containerID="d6c1b27a03ec6a90e0cc59f7180264eb8dee0ede4114765574e02657d5a790ab" exitCode=0 Jul 12 22:12:01.177535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.177486 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerDied","Data":"d6c1b27a03ec6a90e0cc59f7180264eb8dee0ede4114765574e02657d5a790ab"} Jul 12 22:12:01.178751 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.178732 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"0fd8e56312f43688441886be1ed7b71875af3a206ed1d70a6480638aa4a69fcf"} Jul 12 22:12:01.199432 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.199376 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h8dht" podStartSLOduration=3.149667271 podStartE2EDuration="4.199356185s" totalImagesPullingTime="968.59944ms" totalInitContainerRuntime="81.089474ms" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:01.19814155 +0000 UTC m=+57.952394828" watchObservedRunningTime="2026-07-12 22:12:01.199356185 +0000 UTC m=+57.953609442" Jul 12 22:12:01.224663 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:01.224603 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-jx2rn" podStartSLOduration=3.152271497 podStartE2EDuration="4.224586596s" totalImagesPullingTime="1.072315099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:01.223743079 +0000 UTC m=+57.977996337" watchObservedRunningTime="2026-07-12 22:12:01.224586596 +0000 UTC m=+57.978839882" Jul 12 22:12:02.123200 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.123166 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6bb9d9d469-sjdnd"] Jul 12 22:12:02.125867 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.125837 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.128562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.128532 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 12 22:12:02.129437 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.129414 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jmp9f\"" Jul 12 22:12:02.129563 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.129438 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 12 22:12:02.129977 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.129674 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ekrbl5f2t1226\"" Jul 12 22:12:02.129977 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.129704 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 12 22:12:02.130151 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.129987 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 12 22:12:02.141161 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.141128 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6bb9d9d469-sjdnd"] Jul 12 22:12:02.176672 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.176640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:02.192283 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.192249 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f9b9c4c76-cwlpl"] Jul 12 22:12:02.194786 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.194762 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5"] Jul 12 22:12:02.197476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.197367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.201111 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.201038 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 12 22:12:02.201576 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.201554 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5l72r\"" Jul 12 22:12:02.212740 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.212658 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5"] Jul 12 22:12:02.228278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.228227 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c98f9d5cd-9bpln"] Jul 12 22:12:02.230649 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.230623 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.236709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4584f9f2-7823-45a5-821c-b27bd66de9d1-metrics-server-audit-profiles\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.236709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4584f9f2-7823-45a5-821c-b27bd66de9d1-audit-log\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.236709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-secret-metrics-server-tls\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.236709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4584f9f2-7823-45a5-821c-b27bd66de9d1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.236709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97bhc\" (UniqueName: \"kubernetes.io/projected/4584f9f2-7823-45a5-821c-b27bd66de9d1-kube-api-access-97bhc\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.237077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-client-ca-bundle\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.237077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.236784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-secret-metrics-server-client-certs\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.247688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.247118 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c98f9d5cd-9bpln"] Jul 12 22:12:02.283847 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.283813 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:02.338311 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338265 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-serving-cert\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338333 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-oauth-config\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4584f9f2-7823-45a5-821c-b27bd66de9d1-metrics-server-audit-profiles\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.338474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-trusted-ca-bundle\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338413 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4584f9f2-7823-45a5-821c-b27bd66de9d1-audit-log\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.338474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-secret-metrics-server-tls\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.338474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-oauth-serving-cert\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338510 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4584f9f2-7823-45a5-821c-b27bd66de9d1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338536 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-config\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338568 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sxdp\" (UniqueName: \"kubernetes.io/projected/e05c3c61-eb65-451e-aa34-f746f1cf0a08-kube-api-access-7sxdp\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-service-ca\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338635 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97bhc\" (UniqueName: \"kubernetes.io/projected/4584f9f2-7823-45a5-821c-b27bd66de9d1-kube-api-access-97bhc\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338665 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/777e8538-08a1-4a20-8f07-bc9ed5fc28c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-q5jc5\" (UID: \"777e8538-08a1-4a20-8f07-bc9ed5fc28c7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338690 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpl5p\" (UniqueName: \"kubernetes.io/projected/777e8538-08a1-4a20-8f07-bc9ed5fc28c7-kube-api-access-lpl5p\") pod \"monitoring-plugin-5f68df7f58-q5jc5\" (UID: \"777e8538-08a1-4a20-8f07-bc9ed5fc28c7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-client-ca-bundle\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.338790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.338766 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-secret-metrics-server-client-certs\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.339410 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.339382 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4584f9f2-7823-45a5-821c-b27bd66de9d1-audit-log\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.339543 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.339431 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4584f9f2-7823-45a5-821c-b27bd66de9d1-metrics-server-audit-profiles\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.339871 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.339835 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4584f9f2-7823-45a5-821c-b27bd66de9d1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.341747 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.341500 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-secret-metrics-server-tls\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.342490 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.342467 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-client-ca-bundle\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.342599 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.342574 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4584f9f2-7823-45a5-821c-b27bd66de9d1-secret-metrics-server-client-certs\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.350153 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.350095 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97bhc\" (UniqueName: \"kubernetes.io/projected/4584f9f2-7823-45a5-821c-b27bd66de9d1-kube-api-access-97bhc\") pod \"metrics-server-6bb9d9d469-sjdnd\" (UID: \"4584f9f2-7823-45a5-821c-b27bd66de9d1\") " pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.421551 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.421455 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:02.437389 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.437351 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:02.439726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.439697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-serving-cert\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.439726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.439738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-oauth-config\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.439950 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.439765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-trusted-ca-bundle\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.439950 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.439795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-oauth-serving-cert\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.439950 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.439837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-config\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.439950 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.439866 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sxdp\" (UniqueName: \"kubernetes.io/projected/e05c3c61-eb65-451e-aa34-f746f1cf0a08-kube-api-access-7sxdp\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.440163 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.440127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-service-ca\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.440643 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.440180 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/777e8538-08a1-4a20-8f07-bc9ed5fc28c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-q5jc5\" (UID: \"777e8538-08a1-4a20-8f07-bc9ed5fc28c7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.440643 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.440214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpl5p\" (UniqueName: \"kubernetes.io/projected/777e8538-08a1-4a20-8f07-bc9ed5fc28c7-kube-api-access-lpl5p\") pod \"monitoring-plugin-5f68df7f58-q5jc5\" (UID: \"777e8538-08a1-4a20-8f07-bc9ed5fc28c7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.440744 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.440656 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-config\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.441032 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.441002 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-trusted-ca-bundle\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.441145 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.441104 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-oauth-serving-cert\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.441205 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.441178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-service-ca\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.442665 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.442620 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-oauth-config\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.442766 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.442722 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-serving-cert\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.443053 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.443029 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/777e8538-08a1-4a20-8f07-bc9ed5fc28c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-q5jc5\" (UID: \"777e8538-08a1-4a20-8f07-bc9ed5fc28c7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.456289 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.456234 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sxdp\" (UniqueName: \"kubernetes.io/projected/e05c3c61-eb65-451e-aa34-f746f1cf0a08-kube-api-access-7sxdp\") pod \"console-7c98f9d5cd-9bpln\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:02.456427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.456408 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpl5p\" (UniqueName: \"kubernetes.io/projected/777e8538-08a1-4a20-8f07-bc9ed5fc28c7-kube-api-access-lpl5p\") pod \"monitoring-plugin-5f68df7f58-q5jc5\" (UID: \"777e8538-08a1-4a20-8f07-bc9ed5fc28c7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.516758 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.516723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:02.544030 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:02.543978 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:03.130797 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.130768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:03.230870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.230767 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:03.295284 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.295187 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5"] Jul 12 22:12:03.310799 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:03.310762 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777e8538_08a1_4a20_8f07_bc9ed5fc28c7.slice/crio-d029ef5cd13149e6a2c5b122f4992b595fff2c3537606ab04498f268a62d26c0 WatchSource:0}: Error finding container d029ef5cd13149e6a2c5b122f4992b595fff2c3537606ab04498f268a62d26c0: Status 404 returned error can't find the container with id d029ef5cd13149e6a2c5b122f4992b595fff2c3537606ab04498f268a62d26c0 Jul 12 22:12:03.326572 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.326550 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6bb9d9d469-sjdnd"] Jul 12 22:12:03.328456 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:03.328432 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4584f9f2_7823_45a5_821c_b27bd66de9d1.slice/crio-cb8d20a7d567312049ccb90636c5b67329cebefa6573554221d01bc63db53759 WatchSource:0}: Error finding container cb8d20a7d567312049ccb90636c5b67329cebefa6573554221d01bc63db53759: Status 404 returned error can't find the container with id cb8d20a7d567312049ccb90636c5b67329cebefa6573554221d01bc63db53759 Jul 12 22:12:03.345087 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.345024 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c98f9d5cd-9bpln"] Jul 12 22:12:03.349416 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:03.349388 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05c3c61_eb65_451e_aa34_f746f1cf0a08.slice/crio-d7f0c7fd14799d80753b8d211b7e017208b5d683a3e82494f35d71597c35739b WatchSource:0}: Error finding container d7f0c7fd14799d80753b8d211b7e017208b5d683a3e82494f35d71597c35739b: Status 404 returned error can't find the container with id d7f0c7fd14799d80753b8d211b7e017208b5d683a3e82494f35d71597c35739b Jul 12 22:12:03.719799 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.719710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:03.726055 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.726031 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 22:12:03.732516 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.732490 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.735791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.735764 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 12 22:12:03.735938 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.735886 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 12 22:12:03.738537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.738284 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 12 22:12:03.739198 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739176 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 12 22:12:03.739423 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739402 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 12 22:12:03.739585 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739443 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jtt6x\"" Jul 12 22:12:03.739585 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739501 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 12 22:12:03.739897 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739851 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 12 22:12:03.739897 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739882 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 12 22:12:03.740137 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.739894 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 12 22:12:03.740217 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.740139 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 12 22:12:03.740217 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.740152 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7g6a5lar88hqn\"" Jul 12 22:12:03.740375 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.740354 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 12 22:12:03.740476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.740434 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 12 22:12:03.742395 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.742377 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 12 22:12:03.744252 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.744220 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 12 22:12:03.749829 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.749805 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 22:12:03.853378 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853343 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853569 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853569 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853569 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853510 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853569 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffprl\" (UniqueName: \"kubernetes.io/projected/81202aa5-f465-4c08-ab9d-1153df129686-kube-api-access-ffprl\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853720 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853720 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81202aa5-f465-4c08-ab9d-1153df129686-config-out\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853720 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853665 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853720 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853685 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853720 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853716 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81202aa5-f465-4c08-ab9d-1153df129686-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853912 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853737 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853912 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853760 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853912 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853789 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853912 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-config\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.853912 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853839 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.854124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.854124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.853985 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.854124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.854015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.854124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.854038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-web-config\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.854124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.854054 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.954901 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.954858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.954917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.954958 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.954988 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-config\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955062 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955738 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955738 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955675 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955843 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955766 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955917 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955992 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-web-config\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.955992 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.955970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956085 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956085 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956150 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956150 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956224 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffprl\" (UniqueName: \"kubernetes.io/projected/81202aa5-f465-4c08-ab9d-1153df129686-kube-api-access-ffprl\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956224 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956195 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956329 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956202 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956847 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956689 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.956847 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.956752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.958115 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.957526 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.958115 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.957578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81202aa5-f465-4c08-ab9d-1153df129686-config-out\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.958115 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.957610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.958115 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.957634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.958115 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.957692 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81202aa5-f465-4c08-ab9d-1153df129686-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.962619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.959591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.962619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.961187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-config\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.962619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.961356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.962619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.961514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.962619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.962159 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.962925 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.962882 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81202aa5-f465-4c08-ab9d-1153df129686-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.964577 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.964529 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.964946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.964875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81202aa5-f465-4c08-ab9d-1153df129686-config-out\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.965336 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.965130 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.965336 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.965225 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81202aa5-f465-4c08-ab9d-1153df129686-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.965480 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.965352 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.966336 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.966044 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-web-config\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.966939 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.966921 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81202aa5-f465-4c08-ab9d-1153df129686-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:03.972949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:03.972887 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffprl\" (UniqueName: \"kubernetes.io/projected/81202aa5-f465-4c08-ab9d-1153df129686-kube-api-access-ffprl\") pod \"prometheus-k8s-0\" (UID: \"81202aa5-f465-4c08-ab9d-1153df129686\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:04.047733 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.047693 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:04.203153 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.203059 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" event={"ID":"777e8538-08a1-4a20-8f07-bc9ed5fc28c7","Type":"ContainerStarted","Data":"d029ef5cd13149e6a2c5b122f4992b595fff2c3537606ab04498f268a62d26c0"} Jul 12 22:12:04.212083 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.211790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"a16b56a301a484c63aed7f5c9e7de64eacd2a91766296805f89d5bf6283f1b85"} Jul 12 22:12:04.212083 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.211875 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"db8a4ca72018b7cd1e2d32a6b44af123ff3b9706be71b7b6d342f4261557a589"} Jul 12 22:12:04.212083 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.211932 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"ac7b4948aa70d397a6f49c66f87b1c9477f4b96688f63eacdeff78b440310ef7"} Jul 12 22:12:04.212083 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.211974 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"d983d233329f520c67ed6fe8e10824523d62eb747f7977282bf6475fe86813bb"} Jul 12 22:12:04.212083 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.212010 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"3680011c0a959dcad4f5d51954aa5fc7464160e3aed638c12d30eabc8eb05ea2"} Jul 12 22:12:04.217433 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.216550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c98f9d5cd-9bpln" event={"ID":"e05c3c61-eb65-451e-aa34-f746f1cf0a08","Type":"ContainerStarted","Data":"03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c"} Jul 12 22:12:04.217433 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.216601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c98f9d5cd-9bpln" event={"ID":"e05c3c61-eb65-451e-aa34-f746f1cf0a08","Type":"ContainerStarted","Data":"d7f0c7fd14799d80753b8d211b7e017208b5d683a3e82494f35d71597c35739b"} Jul 12 22:12:04.224623 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.224369 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"04f46e52c7af30479a1d43c3bcd7e6f97d3f2e51bce63197f92ce83d56b47dc6"} Jul 12 22:12:04.224623 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.224487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"122e954d315ed2cd56aed5d07ce6dd9223b8eb098d6ccb7366544c6abca7dae6"} Jul 12 22:12:04.224623 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.224532 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"70ec2c67a707c33a460c85ef9c8f6d103e90434aab9c78372ca7c1cbce47b835"} Jul 12 22:12:04.226612 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.226527 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" event={"ID":"4584f9f2-7823-45a5-821c-b27bd66de9d1","Type":"ContainerStarted","Data":"cb8d20a7d567312049ccb90636c5b67329cebefa6573554221d01bc63db53759"} Jul 12 22:12:04.241937 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.241570 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c98f9d5cd-9bpln" podStartSLOduration=2.241550154 podStartE2EDuration="2.241550154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:04.23963318 +0000 UTC m=+60.993886439" watchObservedRunningTime="2026-07-12 22:12:04.241550154 +0000 UTC m=+60.995803412" Jul 12 22:12:04.463815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.463710 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 22:12:04.512002 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:04.511915 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81202aa5_f465_4c08_ab9d_1153df129686.slice/crio-71bb051499b46221396a7a1bbdeb3584c46574003c9eeb3872369b3e945a5c98 WatchSource:0}: Error finding container 71bb051499b46221396a7a1bbdeb3584c46574003c9eeb3872369b3e945a5c98: Status 404 returned error can't find the container with id 71bb051499b46221396a7a1bbdeb3584c46574003c9eeb3872369b3e945a5c98 Jul 12 22:12:04.995258 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:04.995176 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:05.046120 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.046084 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n9zg9" Jul 12 22:12:05.057818 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.057784 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:05.177518 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.177484 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:05.232908 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.232882 2579 generic.go:356] "Generic (PLEG): container finished" podID="81202aa5-f465-4c08-ab9d-1153df129686" containerID="9cbf97b92786e7ee487a6027411a95c9df9074a9340b2e8dff17d206601785bb" exitCode=0 Jul 12 22:12:05.233057 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.232933 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerDied","Data":"9cbf97b92786e7ee487a6027411a95c9df9074a9340b2e8dff17d206601785bb"} Jul 12 22:12:05.233057 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.233007 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"71bb051499b46221396a7a1bbdeb3584c46574003c9eeb3872369b3e945a5c98"} Jul 12 22:12:05.267040 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:05.266954 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:06.238650 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.238612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" event={"ID":"4584f9f2-7823-45a5-821c-b27bd66de9d1","Type":"ContainerStarted","Data":"6767c27ed487d499e9e3a72acbe5a5e5e87c3eaf67aa52d813510f7f20b853ee"} Jul 12 22:12:06.240184 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.240150 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" event={"ID":"777e8538-08a1-4a20-8f07-bc9ed5fc28c7","Type":"ContainerStarted","Data":"15ac90e65a4da6e73d6bd3ebc625a18942e78f6f801d7f1cc71a4130f842a055"} Jul 12 22:12:06.240336 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.240325 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:06.243854 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.243827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbc1e1c4-0292-48b5-bc12-3d61242d4f15","Type":"ContainerStarted","Data":"63360ceb48414466292cbed8e3dc9072c0faa47eed4418f12e526724fc509376"} Jul 12 22:12:06.246204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.246183 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" Jul 12 22:12:06.246773 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.246754 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"cd5fbe00a9999d6a52e69866a502abef16fbeda346f0750525d0a9deb2dc283a"} Jul 12 22:12:06.246842 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.246801 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"0844bad8838e4d015492d74eaf999cb1533edaab9bbfd9e793e4ad6ce3b78380"} Jul 12 22:12:06.246842 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.246827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" event={"ID":"7eae20f3-801a-4c00-837a-b3b62f6f1461","Type":"ContainerStarted","Data":"1a6619005faee77221176f866d85a737b4a9e19f73163496f7d743174daa43b2"} Jul 12 22:12:06.246922 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.246858 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:06.260788 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.260742 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" podStartSLOduration=1.9887942 podStartE2EDuration="4.260726525s" totalImagesPullingTime="2.271932325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:06.259977818 +0000 UTC m=+63.014231076" watchObservedRunningTime="2026-07-12 22:12:06.260726525 +0000 UTC m=+63.014979850" Jul 12 22:12:06.278813 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.278765 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q5jc5" podStartSLOduration=1.995081039 podStartE2EDuration="4.278751051s" totalImagesPullingTime="2.283670012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:06.278226951 +0000 UTC m=+63.032480208" watchObservedRunningTime="2026-07-12 22:12:06.278751051 +0000 UTC m=+63.033004307" Jul 12 22:12:06.304856 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.304797 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" podStartSLOduration=1.979259796 podStartE2EDuration="6.304779384s" totalImagesPullingTime="4.325519588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:12:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:06.302537789 +0000 UTC m=+63.056791048" watchObservedRunningTime="2026-07-12 22:12:06.304779384 +0000 UTC m=+63.059032640" Jul 12 22:12:06.331843 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:06.331784 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.302587819 podStartE2EDuration="8.3317597s" totalImagesPullingTime="5.012406705s" totalInitContainerRuntime="16.765176ms" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:06.329999169 +0000 UTC m=+63.084252429" watchObservedRunningTime="2026-07-12 22:12:06.3317597 +0000 UTC m=+63.086012960" Jul 12 22:12:09.265133 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.265103 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"9e7d213958fec369d1704ddb21e75d423422f2e5bebfd7c679af4a0ef07407ef"} Jul 12 22:12:09.265536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.265168 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"1fd1c6b9e49a7d6757b527f8a011d1e619e286f9f22dac119d6b844c7c642159"} Jul 12 22:12:09.265536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.265195 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"a00f540ae35fc56d2113026244acd0b2d48ad3d9b8a9324339a7dcd5c6a1ae80"} Jul 12 22:12:09.265536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.265218 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"7a05a850b647855207e207590b52aabda8373c6632467847beff912ae21eaa32"} Jul 12 22:12:09.265536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.265269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"6518c0021b231995e0f65205245e008ae5f9874101ee592668ad305d1424edb9"} Jul 12 22:12:09.265536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.265293 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81202aa5-f465-4c08-ab9d-1153df129686","Type":"ContainerStarted","Data":"d9933d0ac67c8e1bbec66d8172662bbe10a028091400ef7b1c4899725ba0ed4c"} Jul 12 22:12:09.299886 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.299817 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.217871716 podStartE2EDuration="6.299797767s" totalImagesPullingTime="3.055279988s" totalInitContainerRuntime="26.646063ms" isStatefulPod=false podCreationTimestamp="2026-07-12 22:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:09.299120019 +0000 UTC m=+66.053373273" watchObservedRunningTime="2026-07-12 22:12:09.299797767 +0000 UTC m=+66.054051026" Jul 12 22:12:09.623603 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.623564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:12:09.623776 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.623648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:12:09.626449 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.626431 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 22:12:09.626583 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.626566 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 22:12:09.637017 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.636986 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a-original-pull-secret\") pod \"global-pull-secret-syncer-nj7st\" (UID: \"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a\") " pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:12:09.637124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.636995 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ae1d9a-b83c-4fa0-adb6-385298d8c5f7-metrics-certs\") pod \"network-metrics-daemon-7c5q2\" (UID: \"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7\") " pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:12:09.671689 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.671660 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npkdh\"" Jul 12 22:12:09.674874 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.674858 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nj7st" Jul 12 22:12:09.679288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.679271 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7c5q2" Jul 12 22:12:09.724383 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.724345 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:12:09.727175 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.727153 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 22:12:09.738360 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.738334 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 22:12:09.747226 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.747197 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:12:09.750967 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.750890 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8ngv\" (UniqueName: \"kubernetes.io/projected/9158ef47-9b2c-4638-ab6d-327ea3602716-kube-api-access-v8ngv\") pod \"network-check-target-8lxs2\" (UID: \"9158ef47-9b2c-4638-ab6d-327ea3602716\") " pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:12:09.754632 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.754564 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:12:09.817036 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.817007 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7c5q2"] Jul 12 22:12:09.820372 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:09.820221 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ae1d9a_b83c_4fa0_adb6_385298d8c5f7.slice/crio-774866e83e1f132a0fab173518d998fb2792c555a0eaa455a9e4410fc781a420 WatchSource:0}: Error finding container 774866e83e1f132a0fab173518d998fb2792c555a0eaa455a9e4410fc781a420: Status 404 returned error can't find the container with id 774866e83e1f132a0fab173518d998fb2792c555a0eaa455a9e4410fc781a420 Jul 12 22:12:09.839900 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.839874 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nj7st"] Jul 12 22:12:09.842890 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:09.842861 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2ca8a7_f8d9_4fc1_a5ea_f9e29c87dd8a.slice/crio-58da81fa9fe7e35753791c2cc37e8e4e4250e14ca6b4daf996e296a8df270a18 WatchSource:0}: Error finding container 58da81fa9fe7e35753791c2cc37e8e4e4250e14ca6b4daf996e296a8df270a18: Status 404 returned error can't find the container with id 58da81fa9fe7e35753791c2cc37e8e4e4250e14ca6b4daf996e296a8df270a18 Jul 12 22:12:09.940156 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.940075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:09.965874 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.965848 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bgjrf\"" Jul 12 22:12:09.973285 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:09.973233 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:12:10.040586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.040557 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:10.099403 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.099380 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lxs2"] Jul 12 22:12:10.101738 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:12:10.101708 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9158ef47_9b2c_4638_ab6d_327ea3602716.slice/crio-fe4154c2fafdb300b94fb783a6f1caf95fac7d95b017323a8cff01cc2824204f WatchSource:0}: Error finding container fe4154c2fafdb300b94fb783a6f1caf95fac7d95b017323a8cff01cc2824204f: Status 404 returned error can't find the container with id fe4154c2fafdb300b94fb783a6f1caf95fac7d95b017323a8cff01cc2824204f Jul 12 22:12:10.164505 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.164457 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:10.265808 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.265708 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:10.270579 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.270539 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nj7st" event={"ID":"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a","Type":"ContainerStarted","Data":"58da81fa9fe7e35753791c2cc37e8e4e4250e14ca6b4daf996e296a8df270a18"} Jul 12 22:12:10.271795 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.271763 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7c5q2" event={"ID":"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7","Type":"ContainerStarted","Data":"774866e83e1f132a0fab173518d998fb2792c555a0eaa455a9e4410fc781a420"} Jul 12 22:12:10.273073 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.273049 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lxs2" event={"ID":"9158ef47-9b2c-4638-ab6d-327ea3602716","Type":"ContainerStarted","Data":"fe4154c2fafdb300b94fb783a6f1caf95fac7d95b017323a8cff01cc2824204f"} Jul 12 22:12:10.332715 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:10.332680 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:11.281102 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:11.281059 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7c5q2" event={"ID":"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7","Type":"ContainerStarted","Data":"b672975be6c13afb2e9928f0cddb87aca8d1f3fd374319d57e79823fafb95d9d"} Jul 12 22:12:11.281596 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:11.281122 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7c5q2" event={"ID":"58ae1d9a-b83c-4fa0-adb6-385298d8c5f7","Type":"ContainerStarted","Data":"d9ba8c29e9a8f61c027b2ad20d8337af2a4d523e9ba53df460a05212805f5dcf"} Jul 12 22:12:11.553643 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:11.553528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:11.647382 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:11.647343 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:11.745218 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:11.745181 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:11.822683 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:11.822606 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:12.259136 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.259058 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56b4ddd8b5-npbnz" Jul 12 22:12:12.279397 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.279363 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:12.296636 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.296183 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7c5q2" podStartSLOduration=67.315326315 podStartE2EDuration="1m8.296162029s" totalImagesPullingTime="980.835714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:11.300991153 +0000 UTC m=+68.055244411" watchObservedRunningTime="2026-07-12 22:12:12.296162029 +0000 UTC m=+69.050415288" Jul 12 22:12:12.376655 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.376619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:12.545112 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.545032 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:12.545344 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.545133 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:12.551519 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:12.551301 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:13.294636 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:13.294607 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:12:13.346781 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:13.346747 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59bc9c9589-q2clp"] Jul 12 22:12:13.472507 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:13.472477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:13.572184 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:13.572089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:13.672651 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:13.672611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:14.048275 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:14.048221 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:12:15.299529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:15.299497 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lxs2" event={"ID":"9158ef47-9b2c-4638-ab6d-327ea3602716","Type":"ContainerStarted","Data":"68b0e98b185429fdaec4b8b398029c96ceb7cd3949e9f203cdbc2a1ab1996ab6"} Jul 12 22:12:15.300054 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:15.299639 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:12:15.300903 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:15.300881 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nj7st" event={"ID":"1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a","Type":"ContainerStarted","Data":"91f932a9f6ffd3e885a683fa98ffc31a8b057d04d029703c145c7081d2b1ebcb"} Jul 12 22:12:15.318903 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:15.318837 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8lxs2" podStartSLOduration=66.56451598 podStartE2EDuration="1m11.318816492s" totalImagesPullingTime="4.754300512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:15.315848124 +0000 UTC m=+72.070101381" watchObservedRunningTime="2026-07-12 22:12:15.318816492 +0000 UTC m=+72.073069751" Jul 12 22:12:15.332574 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:15.332526 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nj7st" podStartSLOduration=66.316409309 podStartE2EDuration="1m11.332512459s" totalImagesPullingTime="5.01610315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:15.331008909 +0000 UTC m=+72.085262178" watchObservedRunningTime="2026-07-12 22:12:15.332512459 +0000 UTC m=+72.086765715" Jul 12 22:12:19.990878 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:19.990848 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:20.089325 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:20.089289 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:22.437812 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:22.437771 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:22.438185 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:22.437841 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:27.221047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.220898 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f9b9c4c76-cwlpl" podUID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" containerName="console" containerID="cri-o://fb822ebaa2e89e58484aedc11bdbe534b02347b997b58b68b26bc9635d334b1d" gracePeriod=15 Jul 12 22:12:27.352434 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.352405 2579 generic.go:356] "Generic (PLEG): container finished" podID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" containerID="fb822ebaa2e89e58484aedc11bdbe534b02347b997b58b68b26bc9635d334b1d" exitCode=0 Jul 12 22:12:27.352554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.352473 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9b9c4c76-cwlpl" event={"ID":"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf","Type":"ContainerDied","Data":"fb822ebaa2e89e58484aedc11bdbe534b02347b997b58b68b26bc9635d334b1d"} Jul 12 22:12:27.464632 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.464606 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:12:27.480443 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.480383 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-serving-cert\") pod \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " Jul 12 22:12:27.480443 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.480429 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2d8f\" (UniqueName: \"kubernetes.io/projected/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-kube-api-access-v2d8f\") pod \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " Jul 12 22:12:27.480593 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.480455 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-oauth-config\") pod \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " Jul 12 22:12:27.480593 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.480558 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-config\") pod \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " Jul 12 22:12:27.480593 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.480589 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-service-ca\") pod \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " Jul 12 22:12:27.480694 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.480662 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-oauth-serving-cert\") pod \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\" (UID: \"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf\") " Jul 12 22:12:27.481057 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.481013 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-config" (OuterVolumeSpecName: "console-config") pod "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" (UID: "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:27.481158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.481015 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-service-ca" (OuterVolumeSpecName: "service-ca") pod "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" (UID: "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:27.481158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.481063 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" (UID: "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:27.482881 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.482855 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" (UID: "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:12:27.483780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.483717 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-kube-api-access-v2d8f" (OuterVolumeSpecName: "kube-api-access-v2d8f") pod "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" (UID: "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf"). InnerVolumeSpecName "kube-api-access-v2d8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:12:27.483780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.483768 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" (UID: "43e36c7f-e5cd-466c-9e58-109cd1cf9bdf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:12:27.581621 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.581586 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-oauth-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:27.581621 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.581616 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:27.581621 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.581625 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2d8f\" (UniqueName: \"kubernetes.io/projected/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-kube-api-access-v2d8f\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:27.581851 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.581634 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-oauth-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:27.581851 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.581643 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-console-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:27.581851 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:27.581650 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:28.357932 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:28.357900 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9b9c4c76-cwlpl" Jul 12 22:12:28.358401 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:28.357900 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9b9c4c76-cwlpl" event={"ID":"43e36c7f-e5cd-466c-9e58-109cd1cf9bdf","Type":"ContainerDied","Data":"5d81164d65ec689f97e5d62e7bf25b46ff6cfd0418e938bd7b3132d152717f40"} Jul 12 22:12:28.358401 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:28.358034 2579 scope.go:117] "RemoveContainer" containerID="fb822ebaa2e89e58484aedc11bdbe534b02347b997b58b68b26bc9635d334b1d" Jul 12 22:12:28.375488 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:28.375463 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f9b9c4c76-cwlpl"] Jul 12 22:12:28.379844 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:28.379822 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f9b9c4c76-cwlpl"] Jul 12 22:12:29.855656 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:29.855619 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" path="/var/lib/kubelet/pods/43e36c7f-e5cd-466c-9e58-109cd1cf9bdf/volumes" Jul 12 22:12:31.387425 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:31.387383 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:31.491697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:31.491652 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:31.586559 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:31.586525 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:34.986585 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:34.986547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:35.054713 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:35.054679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:12:38.377126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.377046 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59bc9c9589-q2clp" podUID="43c0831f-0175-43f3-8744-fa70e923f318" containerName="console" containerID="cri-o://e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18" gracePeriod=15 Jul 12 22:12:38.662990 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.662918 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:12:38.779887 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.779849 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-trusted-ca-bundle\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.779887 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.779894 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-oauth-config\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.780141 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.779919 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-service-ca\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.780141 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.779946 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-serving-cert\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.780141 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.779967 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/43c0831f-0175-43f3-8744-fa70e923f318-kube-api-access-cd9l6\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.780141 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.779993 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-oauth-serving-cert\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.780141 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.780035 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-console-config\") pod \"43c0831f-0175-43f3-8744-fa70e923f318\" (UID: \"43c0831f-0175-43f3-8744-fa70e923f318\") " Jul 12 22:12:38.780509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.780338 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-service-ca" (OuterVolumeSpecName: "service-ca") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:38.780509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.780396 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:38.780627 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.780533 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:38.780627 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.780611 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-console-config" (OuterVolumeSpecName: "console-config") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:38.782156 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.782134 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:12:38.782288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.782216 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c0831f-0175-43f3-8744-fa70e923f318-kube-api-access-cd9l6" (OuterVolumeSpecName: "kube-api-access-cd9l6") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "kube-api-access-cd9l6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:12:38.782288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.782270 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43c0831f-0175-43f3-8744-fa70e923f318" (UID: "43c0831f-0175-43f3-8744-fa70e923f318"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:12:38.881474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881432 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-trusted-ca-bundle\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:38.881474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881465 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-oauth-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:38.881474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881479 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:38.881705 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881492 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c0831f-0175-43f3-8744-fa70e923f318-console-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:38.881705 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881504 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd9l6\" (UniqueName: \"kubernetes.io/projected/43c0831f-0175-43f3-8744-fa70e923f318-kube-api-access-cd9l6\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:38.881705 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881514 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-oauth-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:38.881705 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:38.881526 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c0831f-0175-43f3-8744-fa70e923f318-console-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:12:39.411807 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.411775 2579 generic.go:356] "Generic (PLEG): container finished" podID="43c0831f-0175-43f3-8744-fa70e923f318" containerID="e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18" exitCode=0 Jul 12 22:12:39.412202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.411823 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bc9c9589-q2clp" event={"ID":"43c0831f-0175-43f3-8744-fa70e923f318","Type":"ContainerDied","Data":"e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18"} Jul 12 22:12:39.412202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.411860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bc9c9589-q2clp" event={"ID":"43c0831f-0175-43f3-8744-fa70e923f318","Type":"ContainerDied","Data":"4f7b3fb7449fd47abe16c18da9c4319e272220c87c20fb7e157a88cf0584de31"} Jul 12 22:12:39.412202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.411879 2579 scope.go:117] "RemoveContainer" containerID="e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18" Jul 12 22:12:39.412202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.411825 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bc9c9589-q2clp" Jul 12 22:12:39.420763 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.420747 2579 scope.go:117] "RemoveContainer" containerID="e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18" Jul 12 22:12:39.421024 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:12:39.421003 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18\": container with ID starting with e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18 not found: ID does not exist" containerID="e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18" Jul 12 22:12:39.421088 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.421033 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18"} err="failed to get container status \"e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18\": rpc error: code = NotFound desc = could not find container \"e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18\": container with ID starting with e09a48f246d65b9c2999190c0dfcff82a70356d2a5efccc410e67c2c295f6c18 not found: ID does not exist" Jul 12 22:12:39.434283 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.434254 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59bc9c9589-q2clp"] Jul 12 22:12:39.437994 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.437968 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59bc9c9589-q2clp"] Jul 12 22:12:39.856222 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:39.856183 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c0831f-0175-43f3-8744-fa70e923f318" path="/var/lib/kubelet/pods/43c0831f-0175-43f3-8744-fa70e923f318/volumes" Jul 12 22:12:42.443326 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:42.443298 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:42.447076 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:42.447055 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6bb9d9d469-sjdnd" Jul 12 22:12:46.307905 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:12:46.307875 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8lxs2" Jul 12 22:13:04.048740 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:04.048698 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:13:04.068149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:04.068126 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:13:04.531374 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:04.531346 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 22:13:11.401286 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:11.401227 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:16.881126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:16.881078 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:17.156657 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:17.156567 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:21.956229 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956190 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt"] Jul 12 22:13:21.956736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956716 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c0831f-0175-43f3-8744-fa70e923f318" containerName="console" Jul 12 22:13:21.956736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956735 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c0831f-0175-43f3-8744-fa70e923f318" containerName="console" Jul 12 22:13:21.956848 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956753 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" containerName="console" Jul 12 22:13:21.956848 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956762 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" containerName="console" Jul 12 22:13:21.956848 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956845 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="43e36c7f-e5cd-466c-9e58-109cd1cf9bdf" containerName="console" Jul 12 22:13:21.957006 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.956857 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c0831f-0175-43f3-8744-fa70e923f318" containerName="console" Jul 12 22:13:21.960143 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.960115 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:21.962976 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.962948 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 12 22:13:21.963128 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.963103 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 12 22:13:21.963392 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.963372 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8478p\"" Jul 12 22:13:21.963392 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.963386 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 12 22:13:21.964279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.964261 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 12 22:13:21.964391 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.964313 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 12 22:13:21.969153 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.968891 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 12 22:13:21.978594 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:21.978561 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt"] Jul 12 22:13:22.051200 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051162 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-serving-certs-ca-bundle\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051200 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chvcc\" (UniqueName: \"kubernetes.io/projected/246e296d-fd4b-4ece-8773-db0d25d03299-kube-api-access-chvcc\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051225 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-secret-telemeter-client\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051265 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-federate-client-tls\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-metrics-client-ca\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051533 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051436 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-telemeter-client-tls\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051533 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.051533 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.051497 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152127 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152073 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-serving-certs-ca-bundle\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152127 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chvcc\" (UniqueName: \"kubernetes.io/projected/246e296d-fd4b-4ece-8773-db0d25d03299-kube-api-access-chvcc\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-secret-telemeter-client\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-federate-client-tls\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152207 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-metrics-client-ca\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152318 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-telemeter-client-tls\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152920 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-serving-certs-ca-bundle\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.152995 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.152963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-metrics-client-ca\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.153637 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.153608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246e296d-fd4b-4ece-8773-db0d25d03299-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.154946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.154920 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.155037 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.155020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-secret-telemeter-client\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.155078 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.155058 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-telemeter-client-tls\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.155112 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.155073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/246e296d-fd4b-4ece-8773-db0d25d03299-federate-client-tls\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.161490 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.161472 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chvcc\" (UniqueName: \"kubernetes.io/projected/246e296d-fd4b-4ece-8773-db0d25d03299-kube-api-access-chvcc\") pod \"telemeter-client-7d9747fbc6-jz8nt\" (UID: \"246e296d-fd4b-4ece-8773-db0d25d03299\") " pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.272053 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.271960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" Jul 12 22:13:22.431866 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.431830 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt"] Jul 12 22:13:22.437447 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:13:22.437405 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246e296d_fd4b_4ece_8773_db0d25d03299.slice/crio-3ce02c7d3f09594c611e866e378a8aa107aecf646ae629c9a41d20103e3d7a58 WatchSource:0}: Error finding container 3ce02c7d3f09594c611e866e378a8aa107aecf646ae629c9a41d20103e3d7a58: Status 404 returned error can't find the container with id 3ce02c7d3f09594c611e866e378a8aa107aecf646ae629c9a41d20103e3d7a58 Jul 12 22:13:22.587017 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:22.586980 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" event={"ID":"246e296d-fd4b-4ece-8773-db0d25d03299","Type":"ContainerStarted","Data":"3ce02c7d3f09594c611e866e378a8aa107aecf646ae629c9a41d20103e3d7a58"} Jul 12 22:13:25.602199 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:25.602161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" event={"ID":"246e296d-fd4b-4ece-8773-db0d25d03299","Type":"ContainerStarted","Data":"c3524bf2955c6867031a1ef39e17e939b0f8ef9e291e339319ae01dd643d0b67"} Jul 12 22:13:25.602658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:25.602224 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" event={"ID":"246e296d-fd4b-4ece-8773-db0d25d03299","Type":"ContainerStarted","Data":"1e8bcdef65e4d9b2c87eb52d5ff05bcea9d73904f6ce1626bbd76e3c0b29a1e6"} Jul 12 22:13:25.602658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:25.602266 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" event={"ID":"246e296d-fd4b-4ece-8773-db0d25d03299","Type":"ContainerStarted","Data":"f8cca1e63100491f4c0c6efe3972829161670ef881aae5c5413501ae0c2109f6"} Jul 12 22:13:25.639409 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:25.639356 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d9747fbc6-jz8nt" podStartSLOduration=2.446247188 podStartE2EDuration="4.639337924s" totalImagesPullingTime="2.193090736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:13:25.638043761 +0000 UTC m=+142.392297031" watchObservedRunningTime="2026-07-12 22:13:25.639337924 +0000 UTC m=+142.393591181" Jul 12 22:13:26.329272 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.329224 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b84744d5-x47st"] Jul 12 22:13:26.332650 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.332630 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.346777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.346747 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b84744d5-x47st"] Jul 12 22:13:26.393723 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6lfd\" (UniqueName: \"kubernetes.io/projected/3168662c-c6c9-464f-a743-436160341e0c-kube-api-access-j6lfd\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.393723 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-serving-cert\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.393932 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393759 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-console-config\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.393932 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393816 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-trusted-ca-bundle\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.393932 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-oauth-serving-cert\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.393932 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-service-ca\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.394052 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.393981 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-oauth-config\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.395619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.395590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:26.458782 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.458751 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:26.495155 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-serving-cert\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.495155 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495157 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-console-config\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.495427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495181 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-trusted-ca-bundle\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.495427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-oauth-serving-cert\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.495427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495267 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-service-ca\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.495427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-oauth-config\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.495427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.495382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6lfd\" (UniqueName: \"kubernetes.io/projected/3168662c-c6c9-464f-a743-436160341e0c-kube-api-access-j6lfd\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.496163 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.496123 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-oauth-serving-cert\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.496291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.496161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-console-config\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.496291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.496162 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-service-ca\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.496291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.496266 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-trusted-ca-bundle\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.497738 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.497717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-oauth-config\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.497992 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.497974 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-serving-cert\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.508486 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.508457 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6lfd\" (UniqueName: \"kubernetes.io/projected/3168662c-c6c9-464f-a743-436160341e0c-kube-api-access-j6lfd\") pod \"console-66b84744d5-x47st\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.643876 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.643779 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:26.779990 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:26.779965 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b84744d5-x47st"] Jul 12 22:13:26.782277 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:13:26.782233 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3168662c_c6c9_464f_a743_436160341e0c.slice/crio-9d59babc511e31ad8f6431849b108222191c608d0a6a908c3c15c64fc5509bf2 WatchSource:0}: Error finding container 9d59babc511e31ad8f6431849b108222191c608d0a6a908c3c15c64fc5509bf2: Status 404 returned error can't find the container with id 9d59babc511e31ad8f6431849b108222191c608d0a6a908c3c15c64fc5509bf2 Jul 12 22:13:27.612156 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:27.612117 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b84744d5-x47st" event={"ID":"3168662c-c6c9-464f-a743-436160341e0c","Type":"ContainerStarted","Data":"c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b"} Jul 12 22:13:27.612424 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:27.612195 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b84744d5-x47st" event={"ID":"3168662c-c6c9-464f-a743-436160341e0c","Type":"ContainerStarted","Data":"9d59babc511e31ad8f6431849b108222191c608d0a6a908c3c15c64fc5509bf2"} Jul 12 22:13:27.642424 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:27.642374 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b84744d5-x47st" podStartSLOduration=1.642357423 podStartE2EDuration="1.642357423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:13:27.641433981 +0000 UTC m=+144.395687241" watchObservedRunningTime="2026-07-12 22:13:27.642357423 +0000 UTC m=+144.396610724" Jul 12 22:13:32.105206 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:32.105169 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:32.171544 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:32.171512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:36.643906 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.643863 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:36.644338 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.643946 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:36.649781 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.649756 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:36.653747 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.653725 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:13:36.724736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.724701 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c98f9d5cd-9bpln"] Jul 12 22:13:36.899733 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.899649 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f5fd7469c-qw6qx"] Jul 12 22:13:36.904650 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.904625 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.918032 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.918004 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:36.918820 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.918796 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5fd7469c-qw6qx"] Jul 12 22:13:36.991941 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.991904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-service-ca\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.992124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.991949 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-oauth-config\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.992124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.992058 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-config\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.992124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.992110 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-oauth-serving-cert\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.992231 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.992132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7ql5\" (UniqueName: \"kubernetes.io/projected/cc5cee06-cab7-4fe9-a340-68a78304a3a3-kube-api-access-k7ql5\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.992231 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.992160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-serving-cert\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:36.992341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:36.992228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-trusted-ca-bundle\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.010662 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.010633 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:37.093563 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-serving-cert\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.093563 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-trusted-ca-bundle\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.093806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-service-ca\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.093806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093632 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-oauth-config\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.093806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093672 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-config\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.093806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093717 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-oauth-serving-cert\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.093806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.093742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7ql5\" (UniqueName: \"kubernetes.io/projected/cc5cee06-cab7-4fe9-a340-68a78304a3a3-kube-api-access-k7ql5\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.094594 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.094564 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-service-ca\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.094703 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.094619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-oauth-serving-cert\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.094703 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.094667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-config\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.094776 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.094764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-trusted-ca-bundle\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.096630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.096603 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-serving-cert\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.096716 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.096685 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-oauth-config\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.106204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.106168 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7ql5\" (UniqueName: \"kubernetes.io/projected/cc5cee06-cab7-4fe9-a340-68a78304a3a3-kube-api-access-k7ql5\") pod \"console-6f5fd7469c-qw6qx\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.191517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.191430 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:37.214544 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.214506 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:37.293338 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.293299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:37.356838 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.356810 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5fd7469c-qw6qx"] Jul 12 22:13:37.358913 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:13:37.358883 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5cee06_cab7_4fe9_a340_68a78304a3a3.slice/crio-9619813255d240a28330c90fbb03a17cad4512fd7914fbfcf7047410b47b641c WatchSource:0}: Error finding container 9619813255d240a28330c90fbb03a17cad4512fd7914fbfcf7047410b47b641c: Status 404 returned error can't find the container with id 9619813255d240a28330c90fbb03a17cad4512fd7914fbfcf7047410b47b641c Jul 12 22:13:37.654295 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.654229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5fd7469c-qw6qx" event={"ID":"cc5cee06-cab7-4fe9-a340-68a78304a3a3","Type":"ContainerStarted","Data":"4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e"} Jul 12 22:13:37.654757 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.654310 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5fd7469c-qw6qx" event={"ID":"cc5cee06-cab7-4fe9-a340-68a78304a3a3","Type":"ContainerStarted","Data":"9619813255d240a28330c90fbb03a17cad4512fd7914fbfcf7047410b47b641c"} Jul 12 22:13:37.677463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:37.677406 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f5fd7469c-qw6qx" podStartSLOduration=1.677390398 podStartE2EDuration="1.677390398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:13:37.675440268 +0000 UTC m=+154.429693527" watchObservedRunningTime="2026-07-12 22:13:37.677390398 +0000 UTC m=+154.431643655" Jul 12 22:13:44.073745 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:44.073690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:44.146828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:44.146800 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:47.215061 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.215011 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:47.215497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.215085 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:47.219642 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.219620 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:47.699171 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.699145 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:13:47.756166 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.756134 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b84744d5-x47st"] Jul 12 22:13:47.882622 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.882587 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:47.983828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:47.983737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:48.083877 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:48.083845 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:49.520669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:49.520622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:49.991066 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:49.991039 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:50.095160 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:50.095130 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:50.173774 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:50.173747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:13:52.089535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:13:52.089502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:01.745929 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:01.745823 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c98f9d5cd-9bpln" podUID="e05c3c61-eb65-451e-aa34-f746f1cf0a08" containerName="console" containerID="cri-o://03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c" gracePeriod=15 Jul 12 22:14:01.989596 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:01.989566 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:14:02.027201 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027175 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-serving-cert\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027395 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027209 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-oauth-serving-cert\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027395 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027277 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-config\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027395 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027307 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-oauth-config\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027395 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027339 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-service-ca\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027601 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027438 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-trusted-ca-bundle\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027601 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027475 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sxdp\" (UniqueName: \"kubernetes.io/projected/e05c3c61-eb65-451e-aa34-f746f1cf0a08-kube-api-access-7sxdp\") pod \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\" (UID: \"e05c3c61-eb65-451e-aa34-f746f1cf0a08\") " Jul 12 22:14:02.027710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027688 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-config" (OuterVolumeSpecName: "console-config") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:02.027879 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027817 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-service-ca" (OuterVolumeSpecName: "service-ca") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:02.028001 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027923 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:02.028001 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.027964 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.028222 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.028192 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:02.029833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.029791 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e05c3c61-eb65-451e-aa34-f746f1cf0a08-kube-api-access-7sxdp" (OuterVolumeSpecName: "kube-api-access-7sxdp") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "kube-api-access-7sxdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:14:02.029985 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.029957 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:14:02.030027 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.030007 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e05c3c61-eb65-451e-aa34-f746f1cf0a08" (UID: "e05c3c61-eb65-451e-aa34-f746f1cf0a08"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:14:02.128652 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.128616 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-oauth-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.128652 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.128645 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.128652 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.128656 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-trusted-ca-bundle\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.128652 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.128664 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sxdp\" (UniqueName: \"kubernetes.io/projected/e05c3c61-eb65-451e-aa34-f746f1cf0a08-kube-api-access-7sxdp\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.128906 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.128673 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e05c3c61-eb65-451e-aa34-f746f1cf0a08-console-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.128906 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.128682 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e05c3c61-eb65-451e-aa34-f746f1cf0a08-oauth-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:02.758411 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.758373 2579 generic.go:356] "Generic (PLEG): container finished" podID="e05c3c61-eb65-451e-aa34-f746f1cf0a08" containerID="03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c" exitCode=0 Jul 12 22:14:02.758857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.758422 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c98f9d5cd-9bpln" Jul 12 22:14:02.758857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.758444 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c98f9d5cd-9bpln" event={"ID":"e05c3c61-eb65-451e-aa34-f746f1cf0a08","Type":"ContainerDied","Data":"03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c"} Jul 12 22:14:02.758857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.758484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c98f9d5cd-9bpln" event={"ID":"e05c3c61-eb65-451e-aa34-f746f1cf0a08","Type":"ContainerDied","Data":"d7f0c7fd14799d80753b8d211b7e017208b5d683a3e82494f35d71597c35739b"} Jul 12 22:14:02.758857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.758504 2579 scope.go:117] "RemoveContainer" containerID="03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c" Jul 12 22:14:02.767872 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.767853 2579 scope.go:117] "RemoveContainer" containerID="03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c" Jul 12 22:14:02.768147 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:14:02.768127 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c\": container with ID starting with 03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c not found: ID does not exist" containerID="03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c" Jul 12 22:14:02.768182 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.768156 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c"} err="failed to get container status \"03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c\": rpc error: code = NotFound desc = could not find container \"03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c\": container with ID starting with 03064c71568ca6c8eb362ad53edfb7df81e012586ae8d86dded376ec68822e9c not found: ID does not exist" Jul 12 22:14:02.781436 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.781401 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c98f9d5cd-9bpln"] Jul 12 22:14:02.785369 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:02.785342 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c98f9d5cd-9bpln"] Jul 12 22:14:03.861272 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:03.861222 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e05c3c61-eb65-451e-aa34-f746f1cf0a08" path="/var/lib/kubelet/pods/e05c3c61-eb65-451e-aa34-f746f1cf0a08/volumes" Jul 12 22:14:09.276843 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:09.276807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:09.371645 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:09.371611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:12.781673 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:12.781607 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66b84744d5-x47st" podUID="3168662c-c6c9-464f-a743-436160341e0c" containerName="console" containerID="cri-o://c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b" gracePeriod=15 Jul 12 22:14:13.031906 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.031834 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:14:13.129018 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.128984 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6lfd\" (UniqueName: \"kubernetes.io/projected/3168662c-c6c9-464f-a743-436160341e0c-kube-api-access-j6lfd\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129018 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129027 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-service-ca\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129066 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-oauth-config\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129101 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-trusted-ca-bundle\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129123 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-console-config\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129148 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-serving-cert\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129231 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-oauth-serving-cert\") pod \"3168662c-c6c9-464f-a743-436160341e0c\" (UID: \"3168662c-c6c9-464f-a743-436160341e0c\") " Jul 12 22:14:13.129589 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129551 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:13.129589 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129562 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-service-ca" (OuterVolumeSpecName: "service-ca") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:13.129747 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129559 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-console-config" (OuterVolumeSpecName: "console-config") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:13.129839 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.129782 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:14:13.131386 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.131364 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:14:13.131665 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.131640 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3168662c-c6c9-464f-a743-436160341e0c-kube-api-access-j6lfd" (OuterVolumeSpecName: "kube-api-access-j6lfd") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "kube-api-access-j6lfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:14:13.131665 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.131646 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3168662c-c6c9-464f-a743-436160341e0c" (UID: "3168662c-c6c9-464f-a743-436160341e0c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:14:13.230880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230840 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.230880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230871 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-oauth-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.230880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230882 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-trusted-ca-bundle\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.230880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230890 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-console-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.231134 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230899 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3168662c-c6c9-464f-a743-436160341e0c-console-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.231134 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230908 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3168662c-c6c9-464f-a743-436160341e0c-oauth-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.231134 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.230916 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6lfd\" (UniqueName: \"kubernetes.io/projected/3168662c-c6c9-464f-a743-436160341e0c-kube-api-access-j6lfd\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:14:13.803875 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.803832 2579 generic.go:356] "Generic (PLEG): container finished" podID="3168662c-c6c9-464f-a743-436160341e0c" containerID="c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b" exitCode=0 Jul 12 22:14:13.804295 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.803883 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b84744d5-x47st" Jul 12 22:14:13.804295 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.803919 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b84744d5-x47st" event={"ID":"3168662c-c6c9-464f-a743-436160341e0c","Type":"ContainerDied","Data":"c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b"} Jul 12 22:14:13.804295 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.803973 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b84744d5-x47st" event={"ID":"3168662c-c6c9-464f-a743-436160341e0c","Type":"ContainerDied","Data":"9d59babc511e31ad8f6431849b108222191c608d0a6a908c3c15c64fc5509bf2"} Jul 12 22:14:13.804295 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.803995 2579 scope.go:117] "RemoveContainer" containerID="c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b" Jul 12 22:14:13.813373 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.813355 2579 scope.go:117] "RemoveContainer" containerID="c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b" Jul 12 22:14:13.813632 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:14:13.813611 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b\": container with ID starting with c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b not found: ID does not exist" containerID="c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b" Jul 12 22:14:13.813691 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.813646 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b"} err="failed to get container status \"c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b\": rpc error: code = NotFound desc = could not find container \"c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b\": container with ID starting with c17f69b4635af73c057dea0bc53bf3b25a2652606e4cee45c1da480b4e58e98b not found: ID does not exist" Jul 12 22:14:13.829646 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.829612 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b84744d5-x47st"] Jul 12 22:14:13.833910 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.833886 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66b84744d5-x47st"] Jul 12 22:14:13.856000 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:13.855970 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3168662c-c6c9-464f-a743-436160341e0c" path="/var/lib/kubelet/pods/3168662c-c6c9-464f-a743-436160341e0c/volumes" Jul 12 22:14:31.117164 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:31.117124 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:31.969248 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:31.969200 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:34.935729 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:34.935690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:35.002006 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:35.001972 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:36.709228 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:36.709170 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:36.814124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:36.814092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:37.196379 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:37.196350 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:37.303235 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:37.303199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:39.020730 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:39.020665 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:39.319184 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:39.319143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:41.726302 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:41.726233 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:41.826738 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:41.826696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:44.604139 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604046 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jw46t"] Jul 12 22:14:44.604628 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604606 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3168662c-c6c9-464f-a743-436160341e0c" containerName="console" Jul 12 22:14:44.604697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604632 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3168662c-c6c9-464f-a743-436160341e0c" containerName="console" Jul 12 22:14:44.604697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604660 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05c3c61-eb65-451e-aa34-f746f1cf0a08" containerName="console" Jul 12 22:14:44.604697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604669 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05c3c61-eb65-451e-aa34-f746f1cf0a08" containerName="console" Jul 12 22:14:44.604845 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604761 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05c3c61-eb65-451e-aa34-f746f1cf0a08" containerName="console" Jul 12 22:14:44.604845 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.604779 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3168662c-c6c9-464f-a743-436160341e0c" containerName="console" Jul 12 22:14:44.609202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.609179 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.611552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.611531 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 22:14:44.612556 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.612535 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mjrdz\"" Jul 12 22:14:44.612669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.612538 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 22:14:44.615307 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.615279 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jw46t"] Jul 12 22:14:44.715028 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.714982 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9qw7\" (UniqueName: \"kubernetes.io/projected/3fc2f927-8f68-4761-8c6a-f56770c5fbf6-kube-api-access-x9qw7\") pod \"cert-manager-cainjector-559cdd8c87-jw46t\" (UID: \"3fc2f927-8f68-4761-8c6a-f56770c5fbf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.715028 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.715033 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc2f927-8f68-4761-8c6a-f56770c5fbf6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jw46t\" (UID: \"3fc2f927-8f68-4761-8c6a-f56770c5fbf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.816359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.816317 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc2f927-8f68-4761-8c6a-f56770c5fbf6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jw46t\" (UID: \"3fc2f927-8f68-4761-8c6a-f56770c5fbf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.816504 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.816437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9qw7\" (UniqueName: \"kubernetes.io/projected/3fc2f927-8f68-4761-8c6a-f56770c5fbf6-kube-api-access-x9qw7\") pod \"cert-manager-cainjector-559cdd8c87-jw46t\" (UID: \"3fc2f927-8f68-4761-8c6a-f56770c5fbf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.825560 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.825532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc2f927-8f68-4761-8c6a-f56770c5fbf6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jw46t\" (UID: \"3fc2f927-8f68-4761-8c6a-f56770c5fbf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.825747 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.825725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9qw7\" (UniqueName: \"kubernetes.io/projected/3fc2f927-8f68-4761-8c6a-f56770c5fbf6-kube-api-access-x9qw7\") pod \"cert-manager-cainjector-559cdd8c87-jw46t\" (UID: \"3fc2f927-8f68-4761-8c6a-f56770c5fbf6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:44.929066 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:44.928988 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" Jul 12 22:14:45.068165 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:45.068027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jw46t"] Jul 12 22:14:45.070834 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:14:45.070804 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fc2f927_8f68_4761_8c6a_f56770c5fbf6.slice/crio-bbae9c747815bf35406a81a33893f78867f4bd00e6c4163bbb5f3114c95e0cc0 WatchSource:0}: Error finding container bbae9c747815bf35406a81a33893f78867f4bd00e6c4163bbb5f3114c95e0cc0: Status 404 returned error can't find the container with id bbae9c747815bf35406a81a33893f78867f4bd00e6c4163bbb5f3114c95e0cc0 Jul 12 22:14:45.933904 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:45.933860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" event={"ID":"3fc2f927-8f68-4761-8c6a-f56770c5fbf6","Type":"ContainerStarted","Data":"bbae9c747815bf35406a81a33893f78867f4bd00e6c4163bbb5f3114c95e0cc0"} Jul 12 22:14:46.238140 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:46.238051 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:48.866348 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:48.866312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:48.949007 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:48.948972 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" event={"ID":"3fc2f927-8f68-4761-8c6a-f56770c5fbf6","Type":"ContainerStarted","Data":"6510b57db6d73036a631c6601b4960ca15d12a534e5f2c6acf0a2743ad020084"} Jul 12 22:14:48.969567 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:48.969509 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jw46t" podStartSLOduration=1.970707642 podStartE2EDuration="4.969490422s" totalImagesPullingTime="2.99878278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:14:48.968378414 +0000 UTC m=+225.722631673" watchObservedRunningTime="2026-07-12 22:14:48.969490422 +0000 UTC m=+225.723743679" Jul 12 22:14:48.972322 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:48.972295 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:49.083342 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:49.083299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:50.396411 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.396382 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d"] Jul 12 22:14:50.399815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.399798 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.403715 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.403690 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 22:14:50.403924 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.403690 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qd8p2\"" Jul 12 22:14:50.403924 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.403697 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:14:50.411899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.411873 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d"] Jul 12 22:14:50.473820 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.473785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztjrk\" (UniqueName: \"kubernetes.io/projected/c578efd2-166c-4fc7-b298-0e50c2345358-kube-api-access-ztjrk\") pod \"openshift-lws-operator-fd8ccff4c-rms7d\" (UID: \"c578efd2-166c-4fc7-b298-0e50c2345358\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.474013 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.473867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c578efd2-166c-4fc7-b298-0e50c2345358-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rms7d\" (UID: \"c578efd2-166c-4fc7-b298-0e50c2345358\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.574725 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.574687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztjrk\" (UniqueName: \"kubernetes.io/projected/c578efd2-166c-4fc7-b298-0e50c2345358-kube-api-access-ztjrk\") pod \"openshift-lws-operator-fd8ccff4c-rms7d\" (UID: \"c578efd2-166c-4fc7-b298-0e50c2345358\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.574909 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.574772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c578efd2-166c-4fc7-b298-0e50c2345358-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rms7d\" (UID: \"c578efd2-166c-4fc7-b298-0e50c2345358\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.575108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.575094 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c578efd2-166c-4fc7-b298-0e50c2345358-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rms7d\" (UID: \"c578efd2-166c-4fc7-b298-0e50c2345358\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.584167 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.584143 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztjrk\" (UniqueName: \"kubernetes.io/projected/c578efd2-166c-4fc7-b298-0e50c2345358-kube-api-access-ztjrk\") pod \"openshift-lws-operator-fd8ccff4c-rms7d\" (UID: \"c578efd2-166c-4fc7-b298-0e50c2345358\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.709494 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.709397 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" Jul 12 22:14:50.848208 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.848116 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d"] Jul 12 22:14:50.851050 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:14:50.851024 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc578efd2_166c_4fc7_b298_0e50c2345358.slice/crio-09d4b7f75f58bac30215dedfaeabefc41d8a356f86ede9e71aa2b74313ed50c0 WatchSource:0}: Error finding container 09d4b7f75f58bac30215dedfaeabefc41d8a356f86ede9e71aa2b74313ed50c0: Status 404 returned error can't find the container with id 09d4b7f75f58bac30215dedfaeabefc41d8a356f86ede9e71aa2b74313ed50c0 Jul 12 22:14:50.958587 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:50.958550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" event={"ID":"c578efd2-166c-4fc7-b298-0e50c2345358","Type":"ContainerStarted","Data":"09d4b7f75f58bac30215dedfaeabefc41d8a356f86ede9e71aa2b74313ed50c0"} Jul 12 22:14:53.981131 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:53.981037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" event={"ID":"c578efd2-166c-4fc7-b298-0e50c2345358","Type":"ContainerStarted","Data":"4edd7ca8c80e58205e98f2de3c824d8b72c110693bfe9963e8cb0febb98a4287"} Jul 12 22:14:54.000991 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:54.000930 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rms7d" podStartSLOduration=1.203475934 podStartE2EDuration="4.000911948s" totalImagesPullingTime="2.797436014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:14:53.99832944 +0000 UTC m=+230.752582699" watchObservedRunningTime="2026-07-12 22:14:54.000911948 +0000 UTC m=+230.755165211" Jul 12 22:14:54.729951 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:54.729911 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:54.828634 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:54.828564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:14:56.647637 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:14:56.647586 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:08.171762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:08.171701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:08.276178 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:08.276135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:08.975197 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:08.975162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:09.127597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.127563 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b"] Jul 12 22:15:09.132495 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.132476 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.138227 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.138185 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 22:15:09.138360 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.138330 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bq2zt\"" Jul 12 22:15:09.138481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.138455 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 22:15:09.138605 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.138587 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 22:15:09.151624 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.151594 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:09.162481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.162450 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b"] Jul 12 22:15:09.242995 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.242894 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ac51e41a-aa72-4ef7-b90a-8db803b941c0-manager-config\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.242995 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.242952 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac51e41a-aa72-4ef7-b90a-8db803b941c0-cert\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.243429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.243056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wclcg\" (UniqueName: \"kubernetes.io/projected/ac51e41a-aa72-4ef7-b90a-8db803b941c0-kube-api-access-wclcg\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.243429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.243097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac51e41a-aa72-4ef7-b90a-8db803b941c0-metrics-cert\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.343980 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.343945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wclcg\" (UniqueName: \"kubernetes.io/projected/ac51e41a-aa72-4ef7-b90a-8db803b941c0-kube-api-access-wclcg\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.343980 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.343998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac51e41a-aa72-4ef7-b90a-8db803b941c0-metrics-cert\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.344341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.344068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ac51e41a-aa72-4ef7-b90a-8db803b941c0-manager-config\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.344341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.344097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac51e41a-aa72-4ef7-b90a-8db803b941c0-cert\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.344712 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.344670 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ac51e41a-aa72-4ef7-b90a-8db803b941c0-manager-config\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.346537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.346509 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac51e41a-aa72-4ef7-b90a-8db803b941c0-cert\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.346637 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.346553 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac51e41a-aa72-4ef7-b90a-8db803b941c0-metrics-cert\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.361752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.361723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wclcg\" (UniqueName: \"kubernetes.io/projected/ac51e41a-aa72-4ef7-b90a-8db803b941c0-kube-api-access-wclcg\") pod \"lws-controller-manager-64b6b6c6c7-jc64b\" (UID: \"ac51e41a-aa72-4ef7-b90a-8db803b941c0\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.441957 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.441919 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:09.587468 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.587427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:09.608912 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.608882 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b"] Jul 12 22:15:09.610112 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:15:09.610087 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac51e41a_aa72_4ef7_b90a_8db803b941c0.slice/crio-5d889ccf7c7901f654efa83e2f97a5d117a35f5c5aafbd61d99bdacdc163ca97 WatchSource:0}: Error finding container 5d889ccf7c7901f654efa83e2f97a5d117a35f5c5aafbd61d99bdacdc163ca97: Status 404 returned error can't find the container with id 5d889ccf7c7901f654efa83e2f97a5d117a35f5c5aafbd61d99bdacdc163ca97 Jul 12 22:15:09.723458 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.723419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:09.788814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.788770 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7"] Jul 12 22:15:09.794396 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.794375 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:09.797175 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.797148 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 22:15:09.797342 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.797320 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-smkw7\"" Jul 12 22:15:09.797431 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.797404 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 22:15:09.797511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.797493 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 22:15:09.797946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.797929 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 22:15:09.810771 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.810743 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7"] Jul 12 22:15:09.912668 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.912628 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:09.949847 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.949811 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74b0f45b-e197-4494-ac9e-ec820d7bce7e-webhook-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:09.950020 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.949851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74b0f45b-e197-4494-ac9e-ec820d7bce7e-tmp\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:09.950020 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.949879 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxrwl\" (UniqueName: \"kubernetes.io/projected/74b0f45b-e197-4494-ac9e-ec820d7bce7e-kube-api-access-fxrwl\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:09.950020 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.949908 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/74b0f45b-e197-4494-ac9e-ec820d7bce7e-manifests\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:09.950020 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:09.949927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/74b0f45b-e197-4494-ac9e-ec820d7bce7e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.012203 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.012167 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:10.049734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.049655 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" event={"ID":"ac51e41a-aa72-4ef7-b90a-8db803b941c0","Type":"ContainerStarted","Data":"5d889ccf7c7901f654efa83e2f97a5d117a35f5c5aafbd61d99bdacdc163ca97"} Jul 12 22:15:10.050979 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.050959 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74b0f45b-e197-4494-ac9e-ec820d7bce7e-webhook-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.051042 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.050990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74b0f45b-e197-4494-ac9e-ec820d7bce7e-tmp\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.051042 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.051006 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxrwl\" (UniqueName: \"kubernetes.io/projected/74b0f45b-e197-4494-ac9e-ec820d7bce7e-kube-api-access-fxrwl\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.051042 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.051030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/74b0f45b-e197-4494-ac9e-ec820d7bce7e-manifests\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.051190 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.051054 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/74b0f45b-e197-4494-ac9e-ec820d7bce7e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.051435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.051414 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/74b0f45b-e197-4494-ac9e-ec820d7bce7e-manifests\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.053371 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.053350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74b0f45b-e197-4494-ac9e-ec820d7bce7e-tmp\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.053585 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.053566 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/74b0f45b-e197-4494-ac9e-ec820d7bce7e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.053684 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.053667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74b0f45b-e197-4494-ac9e-ec820d7bce7e-webhook-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.060771 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.060745 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxrwl\" (UniqueName: \"kubernetes.io/projected/74b0f45b-e197-4494-ac9e-ec820d7bce7e-kube-api-access-fxrwl\") pod \"opendatahub-operator-controller-manager-56d7986bbd-rrxr7\" (UID: \"74b0f45b-e197-4494-ac9e-ec820d7bce7e\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.105927 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.105897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:10.121114 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.121090 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:10.250042 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.250007 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:10.261835 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.261807 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7"] Jul 12 22:15:10.264975 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:15:10.264947 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b0f45b_e197_4494_ac9e_ec820d7bce7e.slice/crio-57e4785545241e561e0e80852bd40786270d3251c4dfc564bb3e9d14ef969e50 WatchSource:0}: Error finding container 57e4785545241e561e0e80852bd40786270d3251c4dfc564bb3e9d14ef969e50: Status 404 returned error can't find the container with id 57e4785545241e561e0e80852bd40786270d3251c4dfc564bb3e9d14ef969e50 Jul 12 22:15:10.359282 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.359162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:10.506276 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:10.506223 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:11.060667 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:11.060585 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" event={"ID":"74b0f45b-e197-4494-ac9e-ec820d7bce7e","Type":"ContainerStarted","Data":"57e4785545241e561e0e80852bd40786270d3251c4dfc564bb3e9d14ef969e50"} Jul 12 22:15:12.069200 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:12.069162 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" event={"ID":"ac51e41a-aa72-4ef7-b90a-8db803b941c0","Type":"ContainerStarted","Data":"69dd8ee0aae46bba2520dcbc4a317afeecd70e09f135998cb749668fa2ceefd0"} Jul 12 22:15:12.069668 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:12.069419 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:12.109840 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:12.109728 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" podStartSLOduration=1.217955164 podStartE2EDuration="3.109704799s" totalImagesPullingTime="1.891749635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:12.10581202 +0000 UTC m=+248.860065278" watchObservedRunningTime="2026-07-12 22:15:12.109704799 +0000 UTC m=+248.863958056" Jul 12 22:15:14.081039 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:14.081003 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" event={"ID":"74b0f45b-e197-4494-ac9e-ec820d7bce7e","Type":"ContainerStarted","Data":"727020eff4ce9cae4786e995c49c7b20027720f67ab5d16a3e696dd46902e2f3"} Jul 12 22:15:15.089202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:15.089157 2579 generic.go:356] "Generic (PLEG): container finished" podID="74b0f45b-e197-4494-ac9e-ec820d7bce7e" containerID="727020eff4ce9cae4786e995c49c7b20027720f67ab5d16a3e696dd46902e2f3" exitCode=0 Jul 12 22:15:15.089601 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:15.089257 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" event={"ID":"74b0f45b-e197-4494-ac9e-ec820d7bce7e","Type":"ContainerDied","Data":"727020eff4ce9cae4786e995c49c7b20027720f67ab5d16a3e696dd46902e2f3"} Jul 12 22:15:16.096092 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:16.095993 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" event={"ID":"74b0f45b-e197-4494-ac9e-ec820d7bce7e","Type":"ContainerStarted","Data":"f6d7d54d55b12e97fd11dca16c82534fdca7ba77ab006364e945671732844581"} Jul 12 22:15:16.096574 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:16.096116 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:16.121852 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:16.121799 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" podStartSLOduration=2.330580296 podStartE2EDuration="7.121779443s" totalImagesPullingTime="3.745487144s" totalInitContainerRuntime="1.045712003s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:16.120381169 +0000 UTC m=+252.874634427" watchObservedRunningTime="2026-07-12 22:15:16.121779443 +0000 UTC m=+252.876032699" Jul 12 22:15:23.077866 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:23.077830 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jc64b" Jul 12 22:15:27.103427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:27.103341 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-rrxr7" Jul 12 22:15:28.726202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:28.726170 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:28.899511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:28.899481 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:29.332658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:29.332623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:29.672803 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:29.672715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:29.904148 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:29.904123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:30.040805 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:30.040773 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:31.358586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:31.358550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:31.530746 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:31.530719 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:31.770491 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:31.770407 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:31.910119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:31.910088 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:33.467198 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:33.467165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:37.743181 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:37.743142 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:37.899436 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:37.899401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:38.187801 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.187771 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq"] Jul 12 22:15:38.191175 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.191154 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.194971 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.194948 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 22:15:38.195589 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.195564 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 22:15:38.195701 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.195674 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 22:15:38.197553 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.197527 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zbvsx\"" Jul 12 22:15:38.209822 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.209798 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq"] Jul 12 22:15:38.228288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.228256 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:38.299359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299614 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299554 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299614 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299579 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299614 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299712 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299622 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299712 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.299775 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.299740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqljf\" (UniqueName: \"kubernetes.io/projected/a83df2e9-3e4f-433a-81b5-a7b5241087b9-kube-api-access-jqljf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400379 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400484 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400717 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqljf\" (UniqueName: \"kubernetes.io/projected/a83df2e9-3e4f-433a-81b5-a7b5241087b9-kube-api-access-jqljf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.400958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.400930 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.401134 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.401049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.401134 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.401105 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.401364 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.401346 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.402958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.402936 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.403130 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.403110 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.409455 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.409433 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83df2e9-3e4f-433a-81b5-a7b5241087b9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.411630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.411611 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqljf\" (UniqueName: \"kubernetes.io/projected/a83df2e9-3e4f-433a-81b5-a7b5241087b9-kube-api-access-jqljf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pbpsq\" (UID: \"a83df2e9-3e4f-433a-81b5-a7b5241087b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.473298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.473191 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:38.502181 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.502152 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:38.653688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.653646 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq"] Jul 12 22:15:38.655519 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:15:38.655487 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda83df2e9_3e4f_433a_81b5_a7b5241087b9.slice/crio-ee0eaebc5f286bf4dcbf6a3f3ed55cd915845356d1032eaffaf26945bee82f6b WatchSource:0}: Error finding container ee0eaebc5f286bf4dcbf6a3f3ed55cd915845356d1032eaffaf26945bee82f6b: Status 404 returned error can't find the container with id ee0eaebc5f286bf4dcbf6a3f3ed55cd915845356d1032eaffaf26945bee82f6b Jul 12 22:15:38.703091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.703060 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh"] Jul 12 22:15:38.707780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.707762 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.710697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.710676 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 22:15:38.710697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.710694 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 22:15:38.711130 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.711113 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 22:15:38.711362 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.711345 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sqtxz\"" Jul 12 22:15:38.721042 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.721001 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh"] Jul 12 22:15:38.803568 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.804176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.804176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkkxv\" (UniqueName: \"kubernetes.io/projected/7919b193-f295-488e-8cb3-38ba55a3dbd0-kube-api-access-gkkxv\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.804176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.804176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919b193-f295-488e-8cb3-38ba55a3dbd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.804176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.804176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.803789 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904543 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904510 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904543 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkkxv\" (UniqueName: \"kubernetes.io/projected/7919b193-f295-488e-8cb3-38ba55a3dbd0-kube-api-access-gkkxv\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904714 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.904777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.904739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919b193-f295-488e-8cb3-38ba55a3dbd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.905435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.905410 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.906956 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.906933 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919b193-f295-488e-8cb3-38ba55a3dbd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.907071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.906981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.907300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.907285 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.907347 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.907335 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7919b193-f295-488e-8cb3-38ba55a3dbd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.913079 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.913057 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919b193-f295-488e-8cb3-38ba55a3dbd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.913588 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.913570 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkkxv\" (UniqueName: \"kubernetes.io/projected/7919b193-f295-488e-8cb3-38ba55a3dbd0-kube-api-access-gkkxv\") pod \"istiod-openshift-gateway-94bb8fbfd-l4pbh\" (UID: \"7919b193-f295-488e-8cb3-38ba55a3dbd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:38.993559 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:38.993493 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:39.019517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:39.019483 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:39.176748 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:39.176712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh"] Jul 12 22:15:39.178035 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:15:39.178002 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7919b193_f295_488e_8cb3_38ba55a3dbd0.slice/crio-1df62649d3713a1822864dc0799e420a5ab30af0e8af9135c41c65120aac0de2 WatchSource:0}: Error finding container 1df62649d3713a1822864dc0799e420a5ab30af0e8af9135c41c65120aac0de2: Status 404 returned error can't find the container with id 1df62649d3713a1822864dc0799e420a5ab30af0e8af9135c41c65120aac0de2 Jul 12 22:15:39.203485 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:39.203453 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" event={"ID":"a83df2e9-3e4f-433a-81b5-a7b5241087b9","Type":"ContainerStarted","Data":"ee0eaebc5f286bf4dcbf6a3f3ed55cd915845356d1032eaffaf26945bee82f6b"} Jul 12 22:15:39.204706 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:39.204682 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" event={"ID":"7919b193-f295-488e-8cb3-38ba55a3dbd0","Type":"ContainerStarted","Data":"1df62649d3713a1822864dc0799e420a5ab30af0e8af9135c41c65120aac0de2"} Jul 12 22:15:41.205970 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:41.205930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:41.312903 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:41.312868 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:41.683021 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:41.682970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:41.792288 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:41.792220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:42.398181 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.398145 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:15:42.398488 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.398215 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:15:42.398488 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.398262 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:15:42.398742 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.398720 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:15:42.398804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.398767 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:15:42.532726 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.532690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:42.631146 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.631102 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:42.745705 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:42.745474 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:15:43.229539 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.229504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" event={"ID":"a83df2e9-3e4f-433a-81b5-a7b5241087b9","Type":"ContainerStarted","Data":"22ca8e867c02c2f617df56861ab0ddb2f2c522b341653b93107f7e25cd6a3595"} Jul 12 22:15:43.231097 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.231068 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" event={"ID":"7919b193-f295-488e-8cb3-38ba55a3dbd0","Type":"ContainerStarted","Data":"26f3848a458b4d4cab556b14b356296459eecfd1651709d078e61d432eaec990"} Jul 12 22:15:43.231219 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.231120 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:43.233003 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.232984 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" Jul 12 22:15:43.253607 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.253554 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" podStartSLOduration=1.5132533929999998 podStartE2EDuration="5.253540389s" totalImagesPullingTime="3.740286996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:43.251848395 +0000 UTC m=+280.006101652" watchObservedRunningTime="2026-07-12 22:15:43.253540389 +0000 UTC m=+280.007793646" Jul 12 22:15:43.272827 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.272779 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l4pbh" podStartSLOduration=2.054463074 podStartE2EDuration="5.272764713s" totalImagesPullingTime="3.218301639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:43.27043137 +0000 UTC m=+280.024684641" watchObservedRunningTime="2026-07-12 22:15:43.272764713 +0000 UTC m=+280.027017971" Jul 12 22:15:43.503158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.503074 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:43.507566 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:43.507541 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:44.235468 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:44.235431 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:15:44.236522 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:15:44.236503 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pbpsq" Jul 12 22:16:03.686846 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:03.686816 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:16:03.687391 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:03.686918 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:16:30.275437 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:30.275404 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:16:30.370647 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:30.370615 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:16:30.469786 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:30.469747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:16:42.365381 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:42.365346 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:16:43.543618 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:43.543580 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:16:59.342782 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:59.342686 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:16:59.949652 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:16:59.949605 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:04.748729 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:04.748690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:06.908210 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:06.908179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:07.037378 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:07.037335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:07.412941 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:07.412910 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:09.231176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:09.231143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:10.979975 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:10.979947 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:12.572672 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:12.572634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:13.791828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:13.791796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:15.747549 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:15.747519 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:15.798176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:15.798149 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:16.160666 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:16.160631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:16.236649 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:16.236609 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:19.757044 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:19.757014 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:19.858179 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:19.858147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:20.507721 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:20.507679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:20.736231 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:20.736195 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:22.954587 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:22.954555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:23.053750 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:23.053717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:23.160829 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:23.160797 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:29.414625 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:29.414590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:29.519801 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:29.519761 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:31.746487 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:31.746452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:31.885334 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:31.885301 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:35.160377 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:35.160345 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:35.262311 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:35.262281 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:35.752400 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:35.752364 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:35.853527 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:35.853497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:39.184175 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:39.184134 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:39.274535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:39.274500 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:42.553030 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:42.552997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:42.613421 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:42.613390 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:48.248764 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:48.248722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:17:48.320254 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:17:48.320198 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:03.023466 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.023410 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr"] Jul 12 22:18:03.027017 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.026993 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.029639 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.029576 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tnw4c\"" Jul 12 22:18:03.037965 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.037938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr"] Jul 12 22:18:03.043332 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.043305 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:03.143300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143258 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e17f15-1e28-4280-b369-fb272283881d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e17f15-1e28-4280-b369-fb272283881d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143428 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143449 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxswc\" (UniqueName: \"kubernetes.io/projected/54e17f15-1e28-4280-b369-fb272283881d-kube-api-access-dxswc\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143683 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143518 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.143683 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.143575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e17f15-1e28-4280-b369-fb272283881d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244457 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244457 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244470 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244727 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244727 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxswc\" (UniqueName: \"kubernetes.io/projected/54e17f15-1e28-4280-b369-fb272283881d-kube-api-access-dxswc\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244727 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244552 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244727 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244582 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244727 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244622 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e17f15-1e28-4280-b369-fb272283881d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e17f15-1e28-4280-b369-fb272283881d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e17f15-1e28-4280-b369-fb272283881d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.244978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244940 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.245131 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.244980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.245131 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.245113 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.245269 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.245116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.245608 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.245582 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e17f15-1e28-4280-b369-fb272283881d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.247091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.247067 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e17f15-1e28-4280-b369-fb272283881d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.247309 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.247293 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e17f15-1e28-4280-b369-fb272283881d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.252599 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.252572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e17f15-1e28-4280-b369-fb272283881d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.252702 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.252640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxswc\" (UniqueName: \"kubernetes.io/projected/54e17f15-1e28-4280-b369-fb272283881d-kube-api-access-dxswc\") pod \"maas-default-gateway-openshift-default-56654dcf7-2dtxr\" (UID: \"54e17f15-1e28-4280-b369-fb272283881d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.271752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.271718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:03.340079 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.340033 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:03.478965 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.478931 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr"] Jul 12 22:18:03.482753 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:18:03.482723 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e17f15_1e28_4280_b369_fb272283881d.slice/crio-b8ce41726d9e6685fda5398d03e326a8985872162dcb33f02242324bc8016955 WatchSource:0}: Error finding container b8ce41726d9e6685fda5398d03e326a8985872162dcb33f02242324bc8016955: Status 404 returned error can't find the container with id b8ce41726d9e6685fda5398d03e326a8985872162dcb33f02242324bc8016955 Jul 12 22:18:03.484883 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.484851 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:18:03.484958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.484920 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:18:03.484958 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.484953 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:18:03.885857 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.885815 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" event={"ID":"54e17f15-1e28-4280-b369-fb272283881d","Type":"ContainerStarted","Data":"8be8d096843a1216ecbd73592b0dd816669e2d02566a1477eabca38369204891"} Jul 12 22:18:03.886044 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.885875 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" event={"ID":"54e17f15-1e28-4280-b369-fb272283881d","Type":"ContainerStarted","Data":"b8ce41726d9e6685fda5398d03e326a8985872162dcb33f02242324bc8016955"} Jul 12 22:18:03.908596 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:03.908540 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" podStartSLOduration=0.90852276 podStartE2EDuration="908.52276ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:18:03.906578229 +0000 UTC m=+420.660831487" watchObservedRunningTime="2026-07-12 22:18:03.90852276 +0000 UTC m=+420.662776090" Jul 12 22:18:04.340513 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:04.340472 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:04.345798 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:04.345770 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:04.890822 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:04.890793 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:04.891923 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:04.891897 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:18:19.122999 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.122968 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:19.244368 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.244326 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:19.348342 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.348308 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:19.543182 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.543151 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:18:19.546804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.546779 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.549421 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.549398 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 22:18:19.549538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.549398 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 22:18:19.550277 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.550259 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 22:18:19.550277 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.550268 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qs8cq\"" Jul 12 22:18:19.556108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.556087 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:18:19.594432 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.594394 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:18:19.711283 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.711223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b137d60a-f612-47b2-80b3-caf73c46e6f9-config-file\") pod \"limitador-limitador-54d6d988d5-xp47p\" (UID: \"b137d60a-f612-47b2-80b3-caf73c46e6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.711446 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.711325 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfwpf\" (UniqueName: \"kubernetes.io/projected/b137d60a-f612-47b2-80b3-caf73c46e6f9-kube-api-access-bfwpf\") pod \"limitador-limitador-54d6d988d5-xp47p\" (UID: \"b137d60a-f612-47b2-80b3-caf73c46e6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.812812 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.812722 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b137d60a-f612-47b2-80b3-caf73c46e6f9-config-file\") pod \"limitador-limitador-54d6d988d5-xp47p\" (UID: \"b137d60a-f612-47b2-80b3-caf73c46e6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.812812 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.812798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfwpf\" (UniqueName: \"kubernetes.io/projected/b137d60a-f612-47b2-80b3-caf73c46e6f9-kube-api-access-bfwpf\") pod \"limitador-limitador-54d6d988d5-xp47p\" (UID: \"b137d60a-f612-47b2-80b3-caf73c46e6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.813390 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.813369 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b137d60a-f612-47b2-80b3-caf73c46e6f9-config-file\") pod \"limitador-limitador-54d6d988d5-xp47p\" (UID: \"b137d60a-f612-47b2-80b3-caf73c46e6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.825041 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.825009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfwpf\" (UniqueName: \"kubernetes.io/projected/b137d60a-f612-47b2-80b3-caf73c46e6f9-kube-api-access-bfwpf\") pod \"limitador-limitador-54d6d988d5-xp47p\" (UID: \"b137d60a-f612-47b2-80b3-caf73c46e6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.858532 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.858498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:19.978688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.978660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:19.993950 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.993928 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:18:19.995996 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:18:19.995970 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb137d60a_f612_47b2_80b3_caf73c46e6f9.slice/crio-7adb9f06deca34e9f0a174d20e80ef0090f8c081a16defed25931829f3a1f4ed WatchSource:0}: Error finding container 7adb9f06deca34e9f0a174d20e80ef0090f8c081a16defed25931829f3a1f4ed: Status 404 returned error can't find the container with id 7adb9f06deca34e9f0a174d20e80ef0090f8c081a16defed25931829f3a1f4ed Jul 12 22:18:19.998062 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:19.998039 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:18:20.079420 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:20.079320 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:20.971967 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:20.971864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" event={"ID":"b137d60a-f612-47b2-80b3-caf73c46e6f9","Type":"ContainerStarted","Data":"7adb9f06deca34e9f0a174d20e80ef0090f8c081a16defed25931829f3a1f4ed"} Jul 12 22:18:22.982255 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:22.982198 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" event={"ID":"b137d60a-f612-47b2-80b3-caf73c46e6f9","Type":"ContainerStarted","Data":"cfb0999c29f291fb2e9d27d17acb21bc6679f807f2c3f0b6428ef0a94267ac19"} Jul 12 22:18:22.982636 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:22.982285 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:22.999422 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:22.999366 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" podStartSLOduration=1.574755026 podStartE2EDuration="3.999351006s" totalImagesPullingTime="2.42459598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:18:22.998660143 +0000 UTC m=+439.752913402" watchObservedRunningTime="2026-07-12 22:18:22.999351006 +0000 UTC m=+439.753604262" Jul 12 22:18:32.646946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.646908 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-784f58b8b-hgk7h"] Jul 12 22:18:32.650718 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.650696 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.660510 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.660483 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784f58b8b-hgk7h"] Jul 12 22:18:32.699045 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.698987 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:32.735472 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735432 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55dvj\" (UniqueName: \"kubernetes.io/projected/2a245ac5-cdb6-4539-82af-5c3f84e429ec-kube-api-access-55dvj\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.735640 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735538 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-trusted-ca-bundle\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.735640 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735587 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-service-ca\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.735640 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735615 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-oauth-config\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.735640 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-config\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.735797 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-oauth-serving-cert\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.735797 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.735724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-serving-cert\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.752536 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.752505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:32.836435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55dvj\" (UniqueName: \"kubernetes.io/projected/2a245ac5-cdb6-4539-82af-5c3f84e429ec-kube-api-access-55dvj\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.836899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-trusted-ca-bundle\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.836899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-service-ca\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.836899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-oauth-config\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.836899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-config\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.836899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-oauth-serving-cert\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.836899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.836721 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-serving-cert\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.837401 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.837375 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-oauth-serving-cert\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.837401 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.837401 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-config\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.837565 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.837425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-service-ca\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.837565 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.837449 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a245ac5-cdb6-4539-82af-5c3f84e429ec-trusted-ca-bundle\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.839191 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.839160 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-oauth-config\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.839291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.839254 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a245ac5-cdb6-4539-82af-5c3f84e429ec-console-serving-cert\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.844024 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.843999 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55dvj\" (UniqueName: \"kubernetes.io/projected/2a245ac5-cdb6-4539-82af-5c3f84e429ec-kube-api-access-55dvj\") pod \"console-784f58b8b-hgk7h\" (UID: \"2a245ac5-cdb6-4539-82af-5c3f84e429ec\") " pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:32.960516 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:32.960432 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:33.102454 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:33.102430 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784f58b8b-hgk7h"] Jul 12 22:18:33.105013 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:18:33.104974 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a245ac5_cdb6_4539_82af_5c3f84e429ec.slice/crio-5b607b40b2cbd988ddf1a97de10342130420954b83f524f450f0ed2ba9b4a011 WatchSource:0}: Error finding container 5b607b40b2cbd988ddf1a97de10342130420954b83f524f450f0ed2ba9b4a011: Status 404 returned error can't find the container with id 5b607b40b2cbd988ddf1a97de10342130420954b83f524f450f0ed2ba9b4a011 Jul 12 22:18:33.988266 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:33.988205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xp47p" Jul 12 22:18:34.040147 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:34.040106 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784f58b8b-hgk7h" event={"ID":"2a245ac5-cdb6-4539-82af-5c3f84e429ec","Type":"ContainerStarted","Data":"1263c39f520d87527dd788c2805497ee0d3ed736028f5c308228844b475913e1"} Jul 12 22:18:34.040350 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:34.040163 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784f58b8b-hgk7h" event={"ID":"2a245ac5-cdb6-4539-82af-5c3f84e429ec","Type":"ContainerStarted","Data":"5b607b40b2cbd988ddf1a97de10342130420954b83f524f450f0ed2ba9b4a011"} Jul 12 22:18:36.511146 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:36.511111 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:36.743794 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:36.743756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:37.079413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:37.079373 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:42.961284 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:42.961224 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:42.961699 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:42.961320 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:42.965871 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:42.965852 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:42.985042 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:42.984990 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-784f58b8b-hgk7h" podStartSLOduration=10.984974278 podStartE2EDuration="10.984974278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:18:34.058685286 +0000 UTC m=+450.812938542" watchObservedRunningTime="2026-07-12 22:18:42.984974278 +0000 UTC m=+459.739227534" Jul 12 22:18:43.090816 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:43.090785 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-784f58b8b-hgk7h" Jul 12 22:18:43.136812 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:43.136777 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5fd7469c-qw6qx"] Jul 12 22:18:43.233813 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:43.233697 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:43.283925 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:43.283877 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:43.339233 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:43.339202 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:49.286476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:49.286430 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:49.610729 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:49.610636 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:56.249903 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:56.249872 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:56.350477 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:56.350439 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:56.410943 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:56.410908 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:57.002161 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:57.002122 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:59.523141 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:59.523110 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:18:59.618728 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:18:59.618695 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:08.160196 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.160135 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f5fd7469c-qw6qx" podUID="cc5cee06-cab7-4fe9-a340-68a78304a3a3" containerName="console" containerID="cri-o://4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e" gracePeriod=15 Jul 12 22:19:08.409258 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.409218 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:19:08.491648 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491567 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7ql5\" (UniqueName: \"kubernetes.io/projected/cc5cee06-cab7-4fe9-a340-68a78304a3a3-kube-api-access-k7ql5\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.491648 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491628 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-config\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.491648 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491650 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-oauth-serving-cert\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.491901 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491694 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-serving-cert\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.491901 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491754 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-service-ca\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.491901 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491777 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-oauth-config\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.491901 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.491803 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-trusted-ca-bundle\") pod \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\" (UID: \"cc5cee06-cab7-4fe9-a340-68a78304a3a3\") " Jul 12 22:19:08.492111 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.492087 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:19:08.492357 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.492096 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-config" (OuterVolumeSpecName: "console-config") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:19:08.492471 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.492382 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:19:08.492535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.492479 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:19:08.493874 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.493847 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:19:08.493984 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.493895 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc5cee06-cab7-4fe9-a340-68a78304a3a3-kube-api-access-k7ql5" (OuterVolumeSpecName: "kube-api-access-k7ql5") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "kube-api-access-k7ql5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:19:08.493984 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.493933 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc5cee06-cab7-4fe9-a340-68a78304a3a3" (UID: "cc5cee06-cab7-4fe9-a340-68a78304a3a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:19:08.592860 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592825 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:08.592860 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592857 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:08.592860 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592867 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-oauth-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:08.593091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592877 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-trusted-ca-bundle\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:08.593091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592890 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7ql5\" (UniqueName: \"kubernetes.io/projected/cc5cee06-cab7-4fe9-a340-68a78304a3a3-kube-api-access-k7ql5\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:08.593091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592902 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-console-config\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:08.593091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:08.592910 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc5cee06-cab7-4fe9-a340-68a78304a3a3-oauth-serving-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:19:09.213564 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.213528 2579 generic.go:356] "Generic (PLEG): container finished" podID="cc5cee06-cab7-4fe9-a340-68a78304a3a3" containerID="4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e" exitCode=0 Jul 12 22:19:09.213971 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.213578 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5fd7469c-qw6qx" Jul 12 22:19:09.213971 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.213588 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5fd7469c-qw6qx" event={"ID":"cc5cee06-cab7-4fe9-a340-68a78304a3a3","Type":"ContainerDied","Data":"4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e"} Jul 12 22:19:09.213971 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.213638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5fd7469c-qw6qx" event={"ID":"cc5cee06-cab7-4fe9-a340-68a78304a3a3","Type":"ContainerDied","Data":"9619813255d240a28330c90fbb03a17cad4512fd7914fbfcf7047410b47b641c"} Jul 12 22:19:09.213971 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.213664 2579 scope.go:117] "RemoveContainer" containerID="4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e" Jul 12 22:19:09.222533 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.222515 2579 scope.go:117] "RemoveContainer" containerID="4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e" Jul 12 22:19:09.222808 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:19:09.222785 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e\": container with ID starting with 4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e not found: ID does not exist" containerID="4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e" Jul 12 22:19:09.222900 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.222814 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e"} err="failed to get container status \"4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e\": rpc error: code = NotFound desc = could not find container \"4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e\": container with ID starting with 4f340f30aa661b0ca26d82cef282a32258cbf823c37ab360a55e6c881b45284e not found: ID does not exist" Jul 12 22:19:09.234387 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.234365 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5fd7469c-qw6qx"] Jul 12 22:19:09.239839 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.239813 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f5fd7469c-qw6qx"] Jul 12 22:19:09.857676 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:09.857641 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc5cee06-cab7-4fe9-a340-68a78304a3a3" path="/var/lib/kubelet/pods/cc5cee06-cab7-4fe9-a340-68a78304a3a3/volumes" Jul 12 22:19:15.229661 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:15.229625 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:31.148924 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:31.148890 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:31.246549 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:31.246516 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:31.674082 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:31.674053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:31.848767 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:31.848735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:32.158586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:32.158502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:32.262192 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:32.262160 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:32.298802 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:32.298764 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:19:40.380587 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:40.380550 2579 generic.go:356] "Generic (PLEG): container finished" podID="54e17f15-1e28-4280-b369-fb272283881d" containerID="8be8d096843a1216ecbd73592b0dd816669e2d02566a1477eabca38369204891" exitCode=0 Jul 12 22:19:40.381047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:40.380624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" event={"ID":"54e17f15-1e28-4280-b369-fb272283881d","Type":"ContainerDied","Data":"8be8d096843a1216ecbd73592b0dd816669e2d02566a1477eabca38369204891"} Jul 12 22:19:40.381047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:40.381025 2579 scope.go:117] "RemoveContainer" containerID="8be8d096843a1216ecbd73592b0dd816669e2d02566a1477eabca38369204891" Jul 12 22:19:40.382026 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:40.382000 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:19:40.382081 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:40.382067 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:19:40.382120 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:40.382110 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:19:41.241481 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:41.241449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:41.341158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:41.341129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:41.390315 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:41.390281 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" event={"ID":"54e17f15-1e28-4280-b369-fb272283881d","Type":"ContainerStarted","Data":"c19511b2e762a3cffd6262e4d285c3606590e58a3b97d1f7bff6f580e4b47825"} Jul 12 22:19:42.341173 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:42.341137 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:19:42.341501 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:42.341470 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2dtxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 12 22:19:42.341613 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:42.341527 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" podUID="54e17f15-1e28-4280-b369-fb272283881d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 12 22:19:43.341001 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:43.340561 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2dtxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 12 22:19:43.341001 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:43.340625 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" podUID="54e17f15-1e28-4280-b369-fb272283881d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 12 22:19:44.340988 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:44.340932 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2dtxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 12 22:19:44.341188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:44.341025 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" podUID="54e17f15-1e28-4280-b369-fb272283881d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 12 22:19:45.341226 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:45.341172 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2dtxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 12 22:19:45.341734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:45.341269 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" podUID="54e17f15-1e28-4280-b369-fb272283881d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 12 22:19:46.344548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:46.344507 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:19:46.344956 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:46.344834 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:19:46.345586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:46.345562 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2dtxr" Jul 12 22:19:51.376156 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:51.376112 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:19:51.876987 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:19:51.876944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:20:02.481405 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:02.481312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:03.179453 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:03.179419 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:20:20.401798 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:20.401758 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:20.460132 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:20.460087 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:31.780598 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:31.780549 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:32.988099 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:32.988066 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:20:44.115166 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:44.115129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:45.524580 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:45.524543 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:45.626697 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:45.626650 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:45.780561 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:45.780525 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:20:52.177152 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.177105 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs"] Jul 12 22:20:52.177734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.177709 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5cee06-cab7-4fe9-a340-68a78304a3a3" containerName="console" Jul 12 22:20:52.177734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.177725 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5cee06-cab7-4fe9-a340-68a78304a3a3" containerName="console" Jul 12 22:20:52.177868 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.177846 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5cee06-cab7-4fe9-a340-68a78304a3a3" containerName="console" Jul 12 22:20:52.181464 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.181442 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.184126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.184106 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 22:20:52.185120 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.185099 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 12 22:20:52.185207 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.185151 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 22:20:52.185268 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.185105 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7867w\"" Jul 12 22:20:52.190308 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.190285 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs"] Jul 12 22:20:52.206682 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.206660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:20:52.225049 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.225021 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.225158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.225062 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ade778-943b-4503-b748-baa90d6a19c4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.225158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.225096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fl8f\" (UniqueName: \"kubernetes.io/projected/39ade778-943b-4503-b748-baa90d6a19c4-kube-api-access-6fl8f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.225268 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.225182 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.225268 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.225207 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.225350 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.225326 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.325907 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.325871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.325907 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.325914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326142 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.325994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326142 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.326021 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326142 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.326051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ade778-943b-4503-b748-baa90d6a19c4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326142 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.326108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fl8f\" (UniqueName: \"kubernetes.io/projected/39ade778-943b-4503-b748-baa90d6a19c4-kube-api-access-6fl8f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326399 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.326322 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326399 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.326349 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.326399 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.326392 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.328121 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.328100 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39ade778-943b-4503-b748-baa90d6a19c4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.328476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.328456 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39ade778-943b-4503-b748-baa90d6a19c4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.334319 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.334297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fl8f\" (UniqueName: \"kubernetes.io/projected/39ade778-943b-4503-b748-baa90d6a19c4-kube-api-access-6fl8f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kd8rs\" (UID: \"39ade778-943b-4503-b748-baa90d6a19c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.493464 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.493376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:20:52.623628 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.623535 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs"] Jul 12 22:20:52.626540 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:20:52.626508 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ade778_943b_4503_b748_baa90d6a19c4.slice/crio-5a4b72a28bbdffd10dc769188419d03e272abf48d2566b15a7b2392ddca6c918 WatchSource:0}: Error finding container 5a4b72a28bbdffd10dc769188419d03e272abf48d2566b15a7b2392ddca6c918: Status 404 returned error can't find the container with id 5a4b72a28bbdffd10dc769188419d03e272abf48d2566b15a7b2392ddca6c918 Jul 12 22:20:52.735882 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:52.735846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" event={"ID":"39ade778-943b-4503-b748-baa90d6a19c4","Type":"ContainerStarted","Data":"5a4b72a28bbdffd10dc769188419d03e272abf48d2566b15a7b2392ddca6c918"} Jul 12 22:20:53.742482 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:53.742448 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" event={"ID":"39ade778-943b-4503-b748-baa90d6a19c4","Type":"ContainerStarted","Data":"3d95fd5baf62c576d9a270ff0182b0ee48457105788cbb11ca4c93c9a16a6b8e"} Jul 12 22:20:53.760580 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:53.760521 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" podStartSLOduration=0.764652594 podStartE2EDuration="1.76050323s" totalImagesPullingTime="995.850636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:20:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:20:53.758864325 +0000 UTC m=+590.513117582" watchObservedRunningTime="2026-07-12 22:20:53.76050323 +0000 UTC m=+590.514756532" Jul 12 22:20:54.185603 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:20:54.185568 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:21:02.305550 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:02.305512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:21:02.493768 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:02.493727 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:21:02.493972 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:02.493811 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:21:02.506557 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:02.506523 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:21:02.795978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:02.795949 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kd8rs" Jul 12 22:21:03.719165 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:03.719136 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:21:03.719659 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:03.719454 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:21:04.985011 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:04.984971 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:21:12.607421 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:21:12.607382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:15.559180 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:15.559143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:17.206468 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:17.206429 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:17.305946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:17.305906 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:18.679606 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:18.679570 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:18.779860 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:18.779829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:40.849494 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:40.849449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:22:47.103610 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:47.103572 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:22:57.475036 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:22:57.474951 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:23:07.582431 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:23:07.582396 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:23:16.882545 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:23:16.882508 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:23:28.674483 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:23:28.674442 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:23:30.191377 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:23:30.191337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:23:32.065185 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:23:32.065150 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:23:53.978448 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:23:53.978408 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:24:08.981665 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:08.981627 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:24:12.774254 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:12.774203 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:12.874113 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:12.874082 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:13.002635 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:13.002600 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:18.682371 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:18.682337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:18.742713 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:18.742673 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:18.812464 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:18.812433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:18.914531 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:18.914501 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:19.038769 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:19.038734 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:24.580898 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:24.580865 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:24.641672 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:24.641631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:32.814858 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:32.814814 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:24:40.976308 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:40.976264 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:24:51.380487 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:24:51.380446 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:25:02.178968 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:25:02.178930 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:25:10.974578 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:25:10.974538 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:25:22.382220 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:25:22.382180 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:25:32.690292 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:25:32.690255 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:25:44.381594 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:25:44.381553 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:25:54.978747 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:25:54.978710 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:26:03.753196 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:26:03.753167 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:26:03.754116 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:26:03.754094 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:26:16.680086 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:26:16.680045 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:26:26.382517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:26:26.382479 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:26:48.578516 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:26:48.578479 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:27:00.181113 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:27:00.181075 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:27:15.179948 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:27:15.179909 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:27:26.976692 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:27:26.976596 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:27:42.280194 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:27:42.280156 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:27:55.001511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:27:55.001477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:27:56.184734 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:27:56.184700 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:28:19.054085 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:28:19.054050 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:29:41.279320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:29:41.279283 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:29:45.801029 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:29:45.800993 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:29:53.972337 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:29:53.972297 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:29:54.735112 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:29:54.735062 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:30:03.885887 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:30:03.885847 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:30:16.185565 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:30:16.185523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:30:28.877737 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:30:28.877697 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:30:38.386339 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:30:38.386294 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:30:47.579953 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:30:47.579917 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:30:58.884625 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:30:58.884585 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:31:03.795457 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:31:03.795422 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:31:03.795854 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:31:03.795554 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:31:09.178654 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:31:09.178615 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:31:17.879002 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:31:17.878960 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:31:29.176817 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:31:29.176779 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:31:53.576215 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:31:53.576174 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:00.787294 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:00.787208 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:11.576667 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:11.576630 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:21.772384 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:21.772349 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:32.288376 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:32.288339 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:44.901076 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:44.901036 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:50.774907 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:50.774868 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:32:56.576774 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:32:56.576733 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:33:06.973833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:33:06.973793 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:33:14.984571 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:33:14.984531 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:33:25.687351 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:33:25.687267 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:33:36.676313 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:33:36.676277 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:33:54.892367 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:33:54.892333 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:34:07.381798 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:34:07.381757 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:34:20.181279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:34:20.181224 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:34:32.582463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:34:32.582426 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:34:40.582328 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:34:40.582289 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:34:49.704916 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:34:49.704877 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:34:53.581884 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:34:53.581845 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:35:01.412856 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:01.412771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:06.640437 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:06.640397 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:35:17.877674 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:17.877633 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:35:20.989694 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:20.989660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:27.879583 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:27.879548 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:35:28.615612 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:28.615580 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:28.716879 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:28.716846 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:32.743611 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:32.743566 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:41.004828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:41.004790 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:41.778289 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:41.778251 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:35:46.124925 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:46.124885 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:49.172653 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:49.172608 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:49.273445 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:49.273406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:49.657050 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:49.657012 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:50.886417 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:50.886380 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:35:55.311226 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.311190 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:55.370959 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.370921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:55.487069 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.487035 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj"] Jul 12 22:35:55.490688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.490662 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.493385 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.493364 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cbe7f53b-openshift-default-dockercfg-zkm4f\"" Jul 12 22:35:55.514313 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.514286 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj"] Jul 12 22:35:55.528455 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.528423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:55.545368 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-envoy\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545368 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlk8c\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-kube-api-access-dlk8c\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545393 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-token\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545483 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868284b-74ff-43fe-acfc-38387c1c39c4-istiod-ca-cert\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545508 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-socket\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-credential-socket\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-podinfo\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-data\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.545687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.545674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-certs\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.585581 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.585485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:55.647093 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-envoy\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647354 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlk8c\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-kube-api-access-dlk8c\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647354 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647262 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-token\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647354 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868284b-74ff-43fe-acfc-38387c1c39c4-istiod-ca-cert\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-socket\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647386 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-credential-socket\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647415 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-podinfo\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647535 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-data\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647748 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-certs\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-socket\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647806 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647784 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-credential-socket\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.647916 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.647868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-data\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.648093 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.648070 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-certs\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.648166 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.648149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868284b-74ff-43fe-acfc-38387c1c39c4-istiod-ca-cert\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.649453 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.649430 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-envoy\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.649860 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.649840 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-podinfo\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.655946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.655917 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-token\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.656159 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.656141 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlk8c\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-kube-api-access-dlk8c\") pod \"e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.803028 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.802993 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:55.944653 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.944619 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj"] Jul 12 22:35:55.947789 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:35:55.947760 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode868284b_74ff_43fe_acfc_38387c1c39c4.slice/crio-3b580ea7f851b22ead62311555c72d7284ed829dea72838d5188ea0e003c2660 WatchSource:0}: Error finding container 3b580ea7f851b22ead62311555c72d7284ed829dea72838d5188ea0e003c2660: Status 404 returned error can't find the container with id 3b580ea7f851b22ead62311555c72d7284ed829dea72838d5188ea0e003c2660 Jul 12 22:35:55.950012 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.949977 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:35:55.950090 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.950048 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:35:55.950090 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:55.950079 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:35:56.209659 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.209561 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" event={"ID":"e868284b-74ff-43fe-acfc-38387c1c39c4","Type":"ContainerStarted","Data":"c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1"} Jul 12 22:35:56.209815 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.209671 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" event={"ID":"e868284b-74ff-43fe-acfc-38387c1c39c4","Type":"ContainerStarted","Data":"3b580ea7f851b22ead62311555c72d7284ed829dea72838d5188ea0e003c2660"} Jul 12 22:35:56.229606 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.229551 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podStartSLOduration=1.229534333 podStartE2EDuration="1.229534333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:35:56.227666023 +0000 UTC m=+1492.981919280" watchObservedRunningTime="2026-07-12 22:35:56.229534333 +0000 UTC m=+1492.983787589" Jul 12 22:35:56.628224 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.628193 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:56.709139 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.709112 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:56.774480 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.774442 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:56.803315 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:56.803284 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:57.081484 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:57.081452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:57.206452 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:57.206416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:57.312839 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:57.312806 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:35:57.808511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:57.808480 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:58.221308 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:58.221217 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:58.222329 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:58.222310 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:35:58.857458 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:35:58.857428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:36:03.828180 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:03.828153 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:36:03.828973 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:03.828949 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:36:03.873589 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:03.873554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:36:12.293897 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:12.293859 2579 generic.go:356] "Generic (PLEG): container finished" podID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerID="c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1" exitCode=0 Jul 12 22:36:12.294335 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:12.293934 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" event={"ID":"e868284b-74ff-43fe-acfc-38387c1c39c4","Type":"ContainerDied","Data":"c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1"} Jul 12 22:36:12.294406 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:12.294390 2579 scope.go:117] "RemoveContainer" containerID="c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1" Jul 12 22:36:12.295428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:12.295398 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:36:12.295499 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:12.295469 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:36:12.295577 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:12.295500 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:36:13.221820 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:13.221782 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:36:13.300654 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:13.300621 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" event={"ID":"e868284b-74ff-43fe-acfc-38387c1c39c4","Type":"ContainerStarted","Data":"474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b"} Jul 12 22:36:13.804117 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:13.804078 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:36:13.805357 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:13.805325 2579 patch_prober.go:28] interesting pod/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 22:36:13.805492 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:13.805376 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 22:36:14.803831 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:14.803790 2579 patch_prober.go:28] interesting pod/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 22:36:14.804264 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:14.803852 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 22:36:15.803790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:15.803751 2579 patch_prober.go:28] interesting pod/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 22:36:15.803978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:15.803813 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 22:36:16.804312 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:16.804271 2579 patch_prober.go:28] interesting pod/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 22:36:16.804704 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:16.804334 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 22:36:17.807231 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:17.807198 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:36:17.807603 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:17.807488 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:36:17.808224 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:36:17.808207 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:37:02.231423 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:02.231375 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:05.214732 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:05.214693 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:37:06.880980 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:06.880933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:07.564459 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:07.564416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:07.664662 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:07.664612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:12.669774 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:12.669742 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:12.732811 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:12.732775 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:17.116609 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.116570 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj"] Jul 12 22:37:17.117102 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.116862 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" containerID="cri-o://474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b" gracePeriod=30 Jul 12 22:37:17.130569 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.130534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:17.457760 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.457656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:17.555691 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.555645 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:17.661493 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.661459 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:17.809874 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.809836 2579 patch_prober.go:28] interesting pod/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 22:37:17.810043 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:17.809900 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 22:37:18.879209 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:18.879169 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:19.100748 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:19.100701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:19.475447 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:19.475416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:19.576873 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:19.576842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:19.676586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:19.676551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:22.372901 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.372876 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:37:22.419584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419489 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-podinfo\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419536 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868284b-74ff-43fe-acfc-38387c1c39c4-istiod-ca-cert\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419601 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-envoy\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419626 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-certs\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419654 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlk8c\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-kube-api-access-dlk8c\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419714 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-socket\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419737 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-token\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419759 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-data\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.419804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419786 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-credential-socket\") pod \"e868284b-74ff-43fe-acfc-38387c1c39c4\" (UID: \"e868284b-74ff-43fe-acfc-38387c1c39c4\") " Jul 12 22:37:22.420149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.419970 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e868284b-74ff-43fe-acfc-38387c1c39c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:37:22.420222 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.420200 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:37:22.420347 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.420326 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868284b-74ff-43fe-acfc-38387c1c39c4-istiod-ca-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.420413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.420352 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-credential-socket\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.420413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.420396 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:37:22.421187 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.420970 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:37:22.421187 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.421151 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:37:22.422507 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.422483 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:37:22.422682 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.422644 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:37:22.423030 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.423009 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-kube-api-access-dlk8c" (OuterVolumeSpecName: "kube-api-access-dlk8c") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "kube-api-access-dlk8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:37:22.423924 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.423898 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e868284b-74ff-43fe-acfc-38387c1c39c4" (UID: "e868284b-74ff-43fe-acfc-38387c1c39c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:37:22.521282 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521250 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-podinfo\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.521282 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521279 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-envoy\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.521282 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521288 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-certs\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.521526 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521296 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlk8c\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-kube-api-access-dlk8c\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.521526 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521307 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-workload-socket\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.521526 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521315 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-token\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.521526 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.521324 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868284b-74ff-43fe-acfc-38387c1c39c4-istio-data\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:37:22.641549 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.641511 2579 generic.go:356] "Generic (PLEG): container finished" podID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerID="474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b" exitCode=0 Jul 12 22:37:22.641710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.641567 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" Jul 12 22:37:22.641710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.641577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" event={"ID":"e868284b-74ff-43fe-acfc-38387c1c39c4","Type":"ContainerDied","Data":"474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b"} Jul 12 22:37:22.641710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.641629 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj" event={"ID":"e868284b-74ff-43fe-acfc-38387c1c39c4","Type":"ContainerDied","Data":"3b580ea7f851b22ead62311555c72d7284ed829dea72838d5188ea0e003c2660"} Jul 12 22:37:22.641710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.641661 2579 scope.go:117] "RemoveContainer" containerID="474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b" Jul 12 22:37:22.650817 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.650794 2579 scope.go:117] "RemoveContainer" containerID="c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1" Jul 12 22:37:22.658854 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.658837 2579 scope.go:117] "RemoveContainer" containerID="474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b" Jul 12 22:37:22.659126 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:37:22.659104 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b\": container with ID starting with 474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b not found: ID does not exist" containerID="474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b" Jul 12 22:37:22.659177 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.659134 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b"} err="failed to get container status \"474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b\": rpc error: code = NotFound desc = could not find container \"474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b\": container with ID starting with 474bdfd3b6e49a58ff4705d80cc332a9ed7d11d81f0560d213ef31e691e44f4b not found: ID does not exist" Jul 12 22:37:22.659177 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.659152 2579 scope.go:117] "RemoveContainer" containerID="c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1" Jul 12 22:37:22.659416 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:37:22.659383 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1\": container with ID starting with c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1 not found: ID does not exist" containerID="c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1" Jul 12 22:37:22.659509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.659413 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1"} err="failed to get container status \"c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1\": rpc error: code = NotFound desc = could not find container \"c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1\": container with ID starting with c905f5e4c9b866ddae5532395d39ee2e0e9e6e49bfd1661a0b2398536328efe1 not found: ID does not exist" Jul 12 22:37:22.663591 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.663570 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj"] Jul 12 22:37:22.667427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:22.667407 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cbe7f53b-openshift-default-69fc687b89-h84fj"] Jul 12 22:37:23.443688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:23.443656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:23.500346 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:23.500315 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:37:23.858124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:23.858089 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" path="/var/lib/kubelet/pods/e868284b-74ff-43fe-acfc-38387c1c39c4/volumes" Jul 12 22:37:26.680621 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:26.680584 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:37:26.798710 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:37:26.798676 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:23.563092 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:23.563046 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:23.682787 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:23.682750 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:26.776320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:26.776271 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:38:28.904647 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:28.904603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:29.443707 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:29.443671 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:29.776282 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:29.776189 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:34.963994 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:34.963950 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:35.022722 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:35.022692 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:35.199654 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:35.199621 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:35.260736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:35.260647 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:36.313840 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:36.313806 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:36.405788 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:36.405748 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:36.473555 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:36.473520 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:36.606333 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:36.606225 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:37.063882 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:37.063841 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:37.166114 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:37.166062 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:38.454542 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:38.454233 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:39.375024 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:39.374985 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:38:52.509952 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:52.509910 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:52.610029 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:52.609990 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:52.767525 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:52.767435 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:56.537978 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:56.537944 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:38:57.783541 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:38:57.783503 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:39:02.136927 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:02.136888 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:02.237411 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:02.237374 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:07.568126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.568086 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:07.837641 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.837559 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv"] Jul 12 22:39:07.837988 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.837974 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" Jul 12 22:39:07.837988 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.837989 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" Jul 12 22:39:07.838084 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.838004 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" Jul 12 22:39:07.838084 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.838011 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" Jul 12 22:39:07.838149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.838091 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" Jul 12 22:39:07.838149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.838101 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e868284b-74ff-43fe-acfc-38387c1c39c4" containerName="istio-proxy" Jul 12 22:39:07.841107 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.841084 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.843856 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.843831 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-55d0a565-openshift-default-dockercfg-hzqrt\"" Jul 12 22:39:07.858158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.858131 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv"] Jul 12 22:39:07.864965 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.864940 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:07.938685 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.938576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-credential-socket\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.938685 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.938643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdmmq\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-kube-api-access-tdmmq\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.938715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-envoy\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.938942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-socket\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.939008 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-istio-token\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.939056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-data\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.939093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b452d937-3610-414b-9f14-8274169b1461-istiod-ca-cert\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.939126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-certs\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:07.939524 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:07.939159 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b452d937-3610-414b-9f14-8274169b1461-istio-podinfo\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040485 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-istio-token\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-data\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b452d937-3610-414b-9f14-8274169b1461-istiod-ca-cert\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040552 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-certs\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040577 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b452d937-3610-414b-9f14-8274169b1461-istio-podinfo\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-credential-socket\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.040686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040655 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdmmq\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-kube-api-access-tdmmq\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040723 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-envoy\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-socket\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.040927 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-data\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.041079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-credential-socket\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.041174 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-socket\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.041175 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-certs\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.041423 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.041335 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b452d937-3610-414b-9f14-8274169b1461-istiod-ca-cert\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.043014 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.042991 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-envoy\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.043126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.043107 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b452d937-3610-414b-9f14-8274169b1461-istio-podinfo\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.048543 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.048514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdmmq\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-kube-api-access-tdmmq\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.048656 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.048541 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-istio-token\") pod \"e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.155825 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.155717 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:08.294973 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.294935 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv"] Jul 12 22:39:08.297338 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:39:08.297311 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb452d937_3610_414b_9f14_8274169b1461.slice/crio-3bc4ee1daa9f783437d5b1bf5925e7ac55d959a937fb2a1a4218868ecf138605 WatchSource:0}: Error finding container 3bc4ee1daa9f783437d5b1bf5925e7ac55d959a937fb2a1a4218868ecf138605: Status 404 returned error can't find the container with id 3bc4ee1daa9f783437d5b1bf5925e7ac55d959a937fb2a1a4218868ecf138605 Jul 12 22:39:08.299552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.299520 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:39:08.299651 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.299597 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:39:08.299651 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.299644 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:39:08.987008 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:08.986967 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:09.094094 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.094065 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:09.173303 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.173223 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" event={"ID":"b452d937-3610-414b-9f14-8274169b1461","Type":"ContainerStarted","Data":"5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c"} Jul 12 22:39:09.173489 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.173340 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" event={"ID":"b452d937-3610-414b-9f14-8274169b1461","Type":"ContainerStarted","Data":"3bc4ee1daa9f783437d5b1bf5925e7ac55d959a937fb2a1a4218868ecf138605"} Jul 12 22:39:09.200539 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.200505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:09.201352 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.201311 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" podStartSLOduration=2.201295668 podStartE2EDuration="2.201295668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:39:09.199509938 +0000 UTC m=+1685.953763191" watchObservedRunningTime="2026-07-12 22:39:09.201295668 +0000 UTC m=+1685.955548916" Jul 12 22:39:09.366859 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.366821 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:09.847441 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.847412 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:09.948214 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:09.948181 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:10.156954 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:10.156872 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:10.161752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:10.161728 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:10.178561 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:10.178534 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:10.179501 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:10.179481 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:12.091361 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:12.091322 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:39:13.792046 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:13.792013 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:13.844207 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:13.844067 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:16.953836 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:16.953797 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:39:20.228962 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:20.228871 2579 generic.go:356] "Generic (PLEG): container finished" podID="b452d937-3610-414b-9f14-8274169b1461" containerID="5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c" exitCode=0 Jul 12 22:39:20.228962 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:20.228945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" event={"ID":"b452d937-3610-414b-9f14-8274169b1461","Type":"ContainerDied","Data":"5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c"} Jul 12 22:39:20.229422 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:20.229392 2579 scope.go:117] "RemoveContainer" containerID="5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c" Jul 12 22:39:20.230514 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:20.230483 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:39:20.230584 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:20.230570 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:39:20.230622 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:20.230612 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:39:21.239743 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:21.239701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" event={"ID":"b452d937-3610-414b-9f14-8274169b1461","Type":"ContainerStarted","Data":"616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834"} Jul 12 22:39:22.156491 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:22.156451 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:22.157523 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:22.157495 2579 patch_prober.go:28] interesting pod/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 22:39:22.157640 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:22.157548 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 22:39:23.157120 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:23.157065 2579 patch_prober.go:28] interesting pod/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 22:39:23.157559 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:23.157143 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 22:39:24.156432 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:24.156394 2579 patch_prober.go:28] interesting pod/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 22:39:24.156614 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:24.156453 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 22:39:25.159824 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:25.159745 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:25.160184 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:25.160116 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:39:25.160727 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:39:25.160706 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:40:14.032327 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:14.032294 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:17.080383 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:17.080344 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:40:18.531446 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:18.531414 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:24.214647 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:24.214610 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:24.275454 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:24.275420 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:28.719887 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:28.719848 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv"] Jul 12 22:40:28.720302 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:28.720083 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" containerID="cri-o://616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834" gracePeriod=30 Jul 12 22:40:28.742671 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:28.742632 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:29.384827 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:29.384788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:29.507161 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:29.507123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:31.023219 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:31.023154 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:33.964754 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:33.964729 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:40:34.029539 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029497 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-certs\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.029724 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029571 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b452d937-3610-414b-9f14-8274169b1461-istio-podinfo\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.029724 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029606 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-socket\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.029724 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029682 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-envoy\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.029724 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029715 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-credential-socket\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.029941 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029748 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-istio-token\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.029941 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029763 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:40:34.029941 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029867 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b452d937-3610-414b-9f14-8274169b1461-istiod-ca-cert\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.030120 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.029960 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-data\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.030120 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030068 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdmmq\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-kube-api-access-tdmmq\") pod \"b452d937-3610-414b-9f14-8274169b1461\" (UID: \"b452d937-3610-414b-9f14-8274169b1461\") " Jul 12 22:40:34.030234 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030193 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b452d937-3610-414b-9f14-8274169b1461-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:40:34.030529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030509 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-certs\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.030629 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030534 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b452d937-3610-414b-9f14-8274169b1461-istiod-ca-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.031077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030847 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:40:34.031077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030901 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-data" (OuterVolumeSpecName: "istio-data") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:40:34.031077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.030918 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:40:34.032223 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.032196 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b452d937-3610-414b-9f14-8274169b1461-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:40:34.032757 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.032730 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-kube-api-access-tdmmq" (OuterVolumeSpecName: "kube-api-access-tdmmq") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "kube-api-access-tdmmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:40:34.032862 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.032837 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-istio-token" (OuterVolumeSpecName: "istio-token") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:40:34.033019 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.033002 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b452d937-3610-414b-9f14-8274169b1461" (UID: "b452d937-3610-414b-9f14-8274169b1461"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:40:34.131552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131513 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-envoy\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.131552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131548 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-credential-socket\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.131552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131557 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-istio-token\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.131780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131565 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-istio-data\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.131780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131573 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdmmq\" (UniqueName: \"kubernetes.io/projected/b452d937-3610-414b-9f14-8274169b1461-kube-api-access-tdmmq\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.131780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131581 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b452d937-3610-414b-9f14-8274169b1461-istio-podinfo\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.131780 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.131590 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b452d937-3610-414b-9f14-8274169b1461-workload-socket\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:40:34.607351 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.607316 2579 generic.go:356] "Generic (PLEG): container finished" podID="b452d937-3610-414b-9f14-8274169b1461" containerID="616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834" exitCode=0 Jul 12 22:40:34.607537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.607368 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" Jul 12 22:40:34.607537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.607409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" event={"ID":"b452d937-3610-414b-9f14-8274169b1461","Type":"ContainerDied","Data":"616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834"} Jul 12 22:40:34.607537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.607472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv" event={"ID":"b452d937-3610-414b-9f14-8274169b1461","Type":"ContainerDied","Data":"3bc4ee1daa9f783437d5b1bf5925e7ac55d959a937fb2a1a4218868ecf138605"} Jul 12 22:40:34.607537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.607503 2579 scope.go:117] "RemoveContainer" containerID="616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834" Jul 12 22:40:34.617534 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.617511 2579 scope.go:117] "RemoveContainer" containerID="5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c" Jul 12 22:40:34.629220 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.629201 2579 scope.go:117] "RemoveContainer" containerID="616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834" Jul 12 22:40:34.629586 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:40:34.629562 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834\": container with ID starting with 616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834 not found: ID does not exist" containerID="616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834" Jul 12 22:40:34.629681 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.629598 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834"} err="failed to get container status \"616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834\": rpc error: code = NotFound desc = could not find container \"616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834\": container with ID starting with 616be6331811747ff5d8674438dd4fb77892a459bad7438a1b78813df2aa0834 not found: ID does not exist" Jul 12 22:40:34.629681 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.629623 2579 scope.go:117] "RemoveContainer" containerID="5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c" Jul 12 22:40:34.629882 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:40:34.629862 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c\": container with ID starting with 5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c not found: ID does not exist" containerID="5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c" Jul 12 22:40:34.629926 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.629889 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c"} err="failed to get container status \"5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c\": rpc error: code = NotFound desc = could not find container \"5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c\": container with ID starting with 5b2fff553bd64fe21c31c5b81b0b46645162d750e66a783e0fe6be665b5c802c not found: ID does not exist" Jul 12 22:40:34.630022 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.630006 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv"] Jul 12 22:40:34.633155 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.633131 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-55d0a565-openshift-default-6b5b655bf9-9hltv"] Jul 12 22:40:34.965226 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:34.965192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:35.856911 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:35.856878 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b452d937-3610-414b-9f14-8274169b1461" path="/var/lib/kubelet/pods/b452d937-3610-414b-9f14-8274169b1461/volumes" Jul 12 22:40:37.047130 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:37.047100 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:37.109513 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:37.109482 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:37.182234 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:37.182204 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:37.302205 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:37.302119 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:37.436739 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:37.436706 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:40:37.811127 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:40:37.811092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:03.814400 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:03.814367 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:03.862403 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:03.862371 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:41:03.863768 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:03.863750 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:41:09.698896 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:09.698852 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:09.763619 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:09.763582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:09.852162 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:09.852130 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:09.993741 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:09.993644 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:25.863162 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:25.863127 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:31.470631 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:31.470562 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:31.532002 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:31.531947 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:31.612162 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:31.612129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:31.712149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:31.712118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:42.188108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:42.188067 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:43.005328 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:43.005289 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:48.939655 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:48.939624 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:49.003134 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:49.003098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:49.103165 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:49.103132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:49.232477 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:49.232397 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:49.336986 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:49.336958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:50.092870 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:50.092832 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:50.181865 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:50.181836 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:50.282843 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:50.282803 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:51.896859 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:51.896825 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:52.784589 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:52.784556 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:41:57.992496 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:57.992453 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:58.093021 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:58.092989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:41:58.707709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:41:58.707675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:02.575497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:02.575457 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:42:04.618630 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:04.618595 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:04.682531 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:04.682476 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:04.860231 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:04.860196 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:05.252108 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:05.252076 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:05.500270 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:05.500202 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:08.682797 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:08.682764 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:08.756128 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:08.756099 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:08.830525 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:08.830493 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:08.930502 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:08.930470 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:09.245297 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:09.245235 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:09.343019 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:09.342982 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:13.183927 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:13.183890 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:13.784501 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:13.784463 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:42:13.879303 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:13.879266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:13.972182 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:13.972154 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:14.322284 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:14.322226 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:15.792217 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:15.792186 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:15.858896 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:15.858862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:20.290772 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:20.290737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:20.358990 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:20.358960 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:31.575502 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:31.575464 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:34.812628 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:34.812594 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:42:35.811406 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:35.811368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:41.564429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:41.564391 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:41.664424 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:41.664388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:46.494091 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:46.494055 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:46.738993 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:46.738957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:46.868500 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:46.868465 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:46.968382 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:46.968349 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:47.096442 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:47.096400 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:52.771512 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:52.771474 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:52.878092 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:52.878056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:52.981175 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:52.981144 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:53.169521 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:53.169491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:58.798427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:58.798387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:42:58.895489 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:42:58.895452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:05.684936 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:05.684906 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:06.084594 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:06.084547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:06.203868 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:06.203828 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:08.975371 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:08.975343 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:09.063889 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:09.063844 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:09.133136 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:09.133104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:09.508786 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:09.508756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:09.634049 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:09.634015 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:09.741474 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:09.741442 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:13.482664 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:13.482622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:16.179100 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:16.179058 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:43:24.120006 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:24.119972 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:28.284750 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:28.284709 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:43:41.536792 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:41.536755 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:41.640097 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:41.640065 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:51.073433 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:51.073389 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:43:53.243428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:53.243391 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:53.343056 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:53.343023 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:57.609937 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:57.609860 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:43:57.711101 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:43:57.711070 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:07.500216 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:07.500179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:09.580279 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:09.580192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:11.189669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:11.189620 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:44:12.034335 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:12.034302 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:17.056651 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:17.056619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:53.248604 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:53.248557 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:54.452891 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:54.452856 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:59.868328 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:59.868295 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:44:59.926467 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:44:59.926435 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:04.419837 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:04.419799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:04.544077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:04.544024 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:05.591779 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:05.591701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:05.687486 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:05.687456 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:05.787647 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:05.787613 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:06.496845 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:06.496811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:06.599078 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:06.599043 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:06.696183 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:06.696147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:08.094363 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:08.094332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:11.276339 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:11.276302 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:45:13.661836 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:13.661802 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:13.720547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:13.720511 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:14.345935 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:14.345903 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:20.065884 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:20.065846 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:21.350542 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:21.350505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:21.683355 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:21.683265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:45:30.755428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:30.755392 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:31.094700 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:31.094672 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:52.084649 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:52.084614 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:57.765517 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:57.765475 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:57.826515 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:57.826489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:58.109277 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:58.109230 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:58.505893 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:58.505812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:45:58.691620 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:45:58.691588 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:01.961659 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:01.961625 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:02.040208 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:02.040175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:02.109519 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:02.109487 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:02.305253 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:02.305206 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:02.605869 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:02.605787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:02.705823 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:02.705787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:03.901965 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:03.901937 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:46:03.904063 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:03.904042 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:46:06.461867 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:06.461833 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:06.798181 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:06.798147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:07.109581 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:07.109493 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:07.726198 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:07.726162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:09.980888 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:09.980822 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:10.056796 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:10.056756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:10.118360 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:10.118326 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:10.199529 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:10.199486 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:10.283690 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:10.283649 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:46:10.688891 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:10.688808 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:10.796550 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:10.796514 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:14.460308 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:14.460277 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:24.439923 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:24.439887 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:24.540366 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:24.540333 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:25.377374 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:25.377331 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:46:31.428843 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:31.428806 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:31.529188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:31.529157 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:32.925788 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:32.925752 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:46:39.572265 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:39.572218 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:46:41.004834 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:46:41.004795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:00.708852 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:00.708777 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:04.598235 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:04.598203 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:06.181219 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:06.181180 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:47:28.279552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:28.279514 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:29.289264 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.289203 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:29.390786 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.390757 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd"] Jul 12 22:47:29.391144 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.391132 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" Jul 12 22:47:29.391188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.391146 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" Jul 12 22:47:29.391188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.391163 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" Jul 12 22:47:29.391188 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.391168 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" Jul 12 22:47:29.391315 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.391234 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" Jul 12 22:47:29.394353 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.394336 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:29.396987 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.396960 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 22:47:29.397126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.396960 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xlrdf\"" Jul 12 22:47:29.400840 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.400818 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd"] Jul 12 22:47:29.558438 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.558333 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6250ca57-158c-44d1-8add-7ef1a206a69e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd\" (UID: \"6250ca57-158c-44d1-8add-7ef1a206a69e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:29.658840 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.658804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6250ca57-158c-44d1-8add-7ef1a206a69e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd\" (UID: \"6250ca57-158c-44d1-8add-7ef1a206a69e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:29.659467 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.659443 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6250ca57-158c-44d1-8add-7ef1a206a69e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd\" (UID: \"6250ca57-158c-44d1-8add-7ef1a206a69e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:29.707036 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.706994 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:29.845031 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.845006 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd"] Jul 12 22:47:29.856177 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.856146 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:47:29.882303 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.882271 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:29.983051 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:29.982996 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:30.640308 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:30.640272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" event={"ID":"6250ca57-158c-44d1-8add-7ef1a206a69e","Type":"ContainerStarted","Data":"b830a4c56609eba1e8bf4f0962b579c6369a14e77e3d85f688ef9178e67faec9"} Jul 12 22:47:32.673044 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:32.672999 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:47:33.656217 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:33.656175 2579 generic.go:356] "Generic (PLEG): container finished" podID="6250ca57-158c-44d1-8add-7ef1a206a69e" containerID="c5ad0cfd38a2c519dc690460fc1633caf56125cf36d2af11b6531aad3391bc2c" exitCode=0 Jul 12 22:47:33.656414 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:33.656270 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" event={"ID":"6250ca57-158c-44d1-8add-7ef1a206a69e","Type":"ContainerDied","Data":"c5ad0cfd38a2c519dc690460fc1633caf56125cf36d2af11b6531aad3391bc2c"} Jul 12 22:47:34.790902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:34.790876 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:34.913820 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:34.913720 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6250ca57-158c-44d1-8add-7ef1a206a69e-maas-api-service-ca\") pod \"6250ca57-158c-44d1-8add-7ef1a206a69e\" (UID: \"6250ca57-158c-44d1-8add-7ef1a206a69e\") " Jul 12 22:47:34.914086 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:34.914063 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6250ca57-158c-44d1-8add-7ef1a206a69e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6250ca57-158c-44d1-8add-7ef1a206a69e" (UID: "6250ca57-158c-44d1-8add-7ef1a206a69e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:47:35.015048 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:35.015009 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6250ca57-158c-44d1-8add-7ef1a206a69e-maas-api-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:47:35.668021 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:35.667980 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" Jul 12 22:47:35.668021 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:35.668003 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd" event={"ID":"6250ca57-158c-44d1-8add-7ef1a206a69e","Type":"ContainerDied","Data":"b830a4c56609eba1e8bf4f0962b579c6369a14e77e3d85f688ef9178e67faec9"} Jul 12 22:47:35.668297 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:35.668082 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b830a4c56609eba1e8bf4f0962b579c6369a14e77e3d85f688ef9178e67faec9" Jul 12 22:47:39.822936 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:39.822905 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:42.874786 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:42.874745 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:44.485943 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:44.485905 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:50.110597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:50.110555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:50.169202 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:50.169165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:51.172355 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:51.172320 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:47:55.147077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.147044 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:55.373508 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.373447 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:55.482005 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.481919 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk"] Jul 12 22:47:55.482369 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.482355 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6250ca57-158c-44d1-8add-7ef1a206a69e" containerName="revoke-keys" Jul 12 22:47:55.482427 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.482371 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6250ca57-158c-44d1-8add-7ef1a206a69e" containerName="revoke-keys" Jul 12 22:47:55.482463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.482441 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b452d937-3610-414b-9f14-8274169b1461" containerName="istio-proxy" Jul 12 22:47:55.482463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.482456 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6250ca57-158c-44d1-8add-7ef1a206a69e" containerName="revoke-keys" Jul 12 22:47:55.485621 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.485601 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:55.488158 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.488132 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xlrdf\"" Jul 12 22:47:55.488291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.488166 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 22:47:55.492095 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.492067 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk"] Jul 12 22:47:55.606192 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.606151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66847cd2-6fc7-40fd-9c71-ba73a7266e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk\" (UID: \"66847cd2-6fc7-40fd-9c71-ba73a7266e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:55.706963 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.706918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66847cd2-6fc7-40fd-9c71-ba73a7266e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk\" (UID: \"66847cd2-6fc7-40fd-9c71-ba73a7266e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:55.707583 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.707560 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66847cd2-6fc7-40fd-9c71-ba73a7266e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk\" (UID: \"66847cd2-6fc7-40fd-9c71-ba73a7266e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:55.796773 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.796741 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:55.933618 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.933538 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk"] Jul 12 22:47:55.936267 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:47:55.936219 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66847cd2_6fc7_40fd_9c71_ba73a7266e36.slice/crio-31a97235f9bc58b5c0e4ba5b1629e43810f4c2a33500bb605a614ed0fe2a7534 WatchSource:0}: Error finding container 31a97235f9bc58b5c0e4ba5b1629e43810f4c2a33500bb605a614ed0fe2a7534: Status 404 returned error can't find the container with id 31a97235f9bc58b5c0e4ba5b1629e43810f4c2a33500bb605a614ed0fe2a7534 Jul 12 22:47:55.966929 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:55.966900 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:56.071145 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:56.071061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:47:56.775519 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:56.775487 2579 generic.go:356] "Generic (PLEG): container finished" podID="66847cd2-6fc7-40fd-9c71-ba73a7266e36" containerID="8c7be858977881df0fe1f7740270d8450de5b0e4d5bba4d19b47697986f0228d" exitCode=0 Jul 12 22:47:56.775896 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:56.775534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" event={"ID":"66847cd2-6fc7-40fd-9c71-ba73a7266e36","Type":"ContainerDied","Data":"8c7be858977881df0fe1f7740270d8450de5b0e4d5bba4d19b47697986f0228d"} Jul 12 22:47:56.775896 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:56.775576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" event={"ID":"66847cd2-6fc7-40fd-9c71-ba73a7266e36","Type":"ContainerStarted","Data":"31a97235f9bc58b5c0e4ba5b1629e43810f4c2a33500bb605a614ed0fe2a7534"} Jul 12 22:47:57.917572 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:57.917548 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:58.029882 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:58.029266 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66847cd2-6fc7-40fd-9c71-ba73a7266e36-maas-api-service-ca\") pod \"66847cd2-6fc7-40fd-9c71-ba73a7266e36\" (UID: \"66847cd2-6fc7-40fd-9c71-ba73a7266e36\") " Jul 12 22:47:58.030074 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:58.029883 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66847cd2-6fc7-40fd-9c71-ba73a7266e36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66847cd2-6fc7-40fd-9c71-ba73a7266e36" (UID: "66847cd2-6fc7-40fd-9c71-ba73a7266e36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:47:58.130493 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:58.130398 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66847cd2-6fc7-40fd-9c71-ba73a7266e36-maas-api-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:47:58.787193 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:58.787164 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" Jul 12 22:47:58.787360 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:58.787199 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk" event={"ID":"66847cd2-6fc7-40fd-9c71-ba73a7266e36","Type":"ContainerDied","Data":"31a97235f9bc58b5c0e4ba5b1629e43810f4c2a33500bb605a614ed0fe2a7534"} Jul 12 22:47:58.787360 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:47:58.787259 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31a97235f9bc58b5c0e4ba5b1629e43810f4c2a33500bb605a614ed0fe2a7534" Jul 12 22:48:05.906851 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:05.906806 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:08.880899 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:08.880861 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:48:10.562541 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:10.562505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:16.340736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:16.340697 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:16.442772 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:16.442743 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:21.243884 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:21.243852 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:21.472222 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:21.472191 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:21.823651 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:21.823614 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:22.146332 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:22.146217 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:22.302656 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:22.302622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:24.815798 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:24.815765 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:24.892696 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:24.892663 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:24.996898 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:24.996863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:25.121667 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:25.121553 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:25.554789 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:25.554750 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:25.660902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:25.660866 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:29.296752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:29.296718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:29.733762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:29.733675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:30.001644 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.001542 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2"] Jul 12 22:48:30.002121 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.002101 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66847cd2-6fc7-40fd-9c71-ba73a7266e36" containerName="revoke-keys" Jul 12 22:48:30.002121 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.002121 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="66847cd2-6fc7-40fd-9c71-ba73a7266e36" containerName="revoke-keys" Jul 12 22:48:30.002309 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.002206 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="66847cd2-6fc7-40fd-9c71-ba73a7266e36" containerName="revoke-keys" Jul 12 22:48:30.005949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.005929 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.008674 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.008652 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-189e5d-openshift-default-dockercfg-5k96h\"" Jul 12 22:48:30.017902 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.017876 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2"] Jul 12 22:48:30.035011 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.034965 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:30.054071 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054040 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-envoy\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054345 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054079 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-credential-socket\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054345 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-podinfo\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054345 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054177 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-socket\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054345 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054280 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-token\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054345 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054342 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istiod-ca-cert\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054607 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4kxm\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-kube-api-access-v4kxm\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054607 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054408 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-data\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.054607 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.054448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-certs\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155528 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155468 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istiod-ca-cert\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155528 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155535 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4kxm\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-kube-api-access-v4kxm\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-data\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155604 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-certs\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-envoy\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155718 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-credential-socket\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-podinfo\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.155814 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-socket\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.156124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-token\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.156124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.155999 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-data\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.156124 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.156061 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-certs\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.156361 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.156341 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istiod-ca-cert\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.156413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.156396 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-credential-socket\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.156456 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.156405 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-socket\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.158225 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.158199 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-envoy\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.158353 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.158310 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-podinfo\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.164063 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.164038 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4kxm\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-kube-api-access-v4kxm\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.164063 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.164053 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-token\") pod \"e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.317224 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.317177 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:30.463101 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.463072 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2"] Jul 12 22:48:30.464907 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:48:30.464879 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1304b457_a93e_48e5_9933_ffb3d4a6ac4f.slice/crio-244f2e4f7a52075c0199084af24d6e3673cc3a1a097a337a043907580b6e9c83 WatchSource:0}: Error finding container 244f2e4f7a52075c0199084af24d6e3673cc3a1a097a337a043907580b6e9c83: Status 404 returned error can't find the container with id 244f2e4f7a52075c0199084af24d6e3673cc3a1a097a337a043907580b6e9c83 Jul 12 22:48:30.467184 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.467153 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:48:30.467270 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.467220 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:48:30.467314 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.467284 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:48:30.832389 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.832348 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:30.955841 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.955804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" event={"ID":"1304b457-a93e-48e5-9933-ffb3d4a6ac4f","Type":"ContainerStarted","Data":"18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00"} Jul 12 22:48:30.956033 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.955895 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" event={"ID":"1304b457-a93e-48e5-9933-ffb3d4a6ac4f","Type":"ContainerStarted","Data":"244f2e4f7a52075c0199084af24d6e3673cc3a1a097a337a043907580b6e9c83"} Jul 12 22:48:30.977335 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:30.977284 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podStartSLOduration=1.977266714 podStartE2EDuration="1.977266714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:48:30.975329923 +0000 UTC m=+2247.729583181" watchObservedRunningTime="2026-07-12 22:48:30.977266714 +0000 UTC m=+2247.731519968" Jul 12 22:48:31.317602 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:31.317556 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:31.322769 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:31.322737 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:31.876507 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:31.876469 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:48:31.960984 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:31.960954 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:31.962051 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:31.962032 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:32.909610 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:32.909578 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:33.008800 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:33.008770 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:33.140704 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:33.140668 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:33.351328 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:33.351293 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:33.723680 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:33.723594 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:33.824126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:33.824095 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:37.419577 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:37.419542 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:46.130342 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:46.130227 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:46.229292 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:46.229254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:49.942847 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:48:49.942806 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@3-27171-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1304b457_a93e_48e5_9933_ffb3d4a6ac4f.slice/crio-18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1304b457_a93e_48e5_9933_ffb3d4a6ac4f.slice/crio-conmon-18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00.scope\": RecentStats: unable to find data in memory cache]" Jul 12 22:48:49.981658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:49.981624 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:48:50.061762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:50.061673 2579 generic.go:356] "Generic (PLEG): container finished" podID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerID="18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00" exitCode=0 Jul 12 22:48:50.061762 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:50.061749 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" event={"ID":"1304b457-a93e-48e5-9933-ffb3d4a6ac4f","Type":"ContainerDied","Data":"18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00"} Jul 12 22:48:50.062196 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:50.062181 2579 scope.go:117] "RemoveContainer" containerID="18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00" Jul 12 22:48:50.063306 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:50.063280 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:48:50.063375 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:50.063343 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:48:50.063416 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:50.063374 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 22:48:51.069081 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:51.069039 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" event={"ID":"1304b457-a93e-48e5-9933-ffb3d4a6ac4f","Type":"ContainerStarted","Data":"815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c"} Jul 12 22:48:51.317731 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:51.317691 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:51.318047 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:51.318025 2579 patch_prober.go:28] interesting pod/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:48:51.318101 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:51.318073 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:48:52.318586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:52.318544 2579 patch_prober.go:28] interesting pod/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:48:52.318974 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:52.318620 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:48:53.318314 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:53.318275 2579 patch_prober.go:28] interesting pod/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:48:53.318497 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:53.318346 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:48:53.955795 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:53.955752 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:54.057647 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:54.057608 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:54.318117 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:54.318084 2579 patch_prober.go:28] interesting pod/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:48:54.318313 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:54.318150 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:48:54.773103 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:54.772994 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:54.874125 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:54.874060 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:55.321854 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:55.321822 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:55.322290 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:55.322266 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:55.322921 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:55.322902 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:48:56.076092 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:56.076058 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:56.178609 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:56.178576 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:56.334883 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:56.334776 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:48:59.678858 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:48:59.678818 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:49:00.615884 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:00.615844 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:06.386758 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:06.386723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:06.486876 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:06.486842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:11.294390 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:11.294356 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:11.517833 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:11.517796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:12.287802 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:12.287767 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:12.385979 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:12.385946 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:22.072741 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:22.072698 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:25.778736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:25.778697 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:49:26.726755 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:26.726701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:32.556085 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:32.556037 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:32.643484 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:32.643430 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:37.405496 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:37.405462 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2"] Jul 12 22:49:37.405949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:37.405715 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" containerID="cri-o://815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c" gracePeriod=30 Jul 12 22:49:37.425548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:37.425507 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:37.643849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:37.643816 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:37.981397 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:37.981364 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:38.407319 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:38.407284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:38.576555 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:38.576518 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:40.325013 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:40.324976 2579 patch_prober.go:28] interesting pod/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 12 22:49:40.325495 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:40.325055 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 22:49:41.239405 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:41.239370 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:41.314314 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:41.314228 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:41.380596 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:41.380564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:41.603658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:41.603622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:41.784874 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:41.784841 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:41.889775 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:41.889683 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:42.664946 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.664876 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:49:42.832056 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832024 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-podinfo\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832056 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832062 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-certs\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832085 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-token\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832113 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-envoy\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832193 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4kxm\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-kube-api-access-v4kxm\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832231 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-credential-socket\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832287 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-socket\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832320 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832311 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istiod-ca-cert\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832631 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832336 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-data\") pod \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\" (UID: \"1304b457-a93e-48e5-9933-ffb3d4a6ac4f\") " Jul 12 22:49:42.832631 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832340 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:49:42.832631 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832557 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:49:42.832794 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832712 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-certs\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.832794 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832731 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-workload-socket\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.832794 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832738 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:49:42.832794 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.832778 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:49:42.833323 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.833295 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:49:42.834505 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.834470 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-kube-api-access-v4kxm" (OuterVolumeSpecName: "kube-api-access-v4kxm") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "kube-api-access-v4kxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:49:42.834601 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.834569 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:49:42.834776 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.834747 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:49:42.834828 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.834753 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "1304b457-a93e-48e5-9933-ffb3d4a6ac4f" (UID: "1304b457-a93e-48e5-9933-ffb3d4a6ac4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933751 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4kxm\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-kube-api-access-v4kxm\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933783 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-credential-socket\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933797 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istiod-ca-cert\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933807 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-data\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933815 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-podinfo\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933822 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-token\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:42.933849 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:42.933830 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1304b457-a93e-48e5-9933-ffb3d4a6ac4f-istio-envoy\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:49:43.330269 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.330208 2579 generic.go:356] "Generic (PLEG): container finished" podID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerID="815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c" exitCode=0 Jul 12 22:49:43.330428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.330279 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" Jul 12 22:49:43.330428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.330301 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" event={"ID":"1304b457-a93e-48e5-9933-ffb3d4a6ac4f","Type":"ContainerDied","Data":"815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c"} Jul 12 22:49:43.330428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.330355 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2" event={"ID":"1304b457-a93e-48e5-9933-ffb3d4a6ac4f","Type":"ContainerDied","Data":"244f2e4f7a52075c0199084af24d6e3673cc3a1a097a337a043907580b6e9c83"} Jul 12 22:49:43.330428 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.330377 2579 scope.go:117] "RemoveContainer" containerID="815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c" Jul 12 22:49:43.339686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.339671 2579 scope.go:117] "RemoveContainer" containerID="18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00" Jul 12 22:49:43.350402 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.350376 2579 scope.go:117] "RemoveContainer" containerID="815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c" Jul 12 22:49:43.350901 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:49:43.350881 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c\": container with ID starting with 815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c not found: ID does not exist" containerID="815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c" Jul 12 22:49:43.350988 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.350913 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c"} err="failed to get container status \"815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c\": rpc error: code = NotFound desc = could not find container \"815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c\": container with ID starting with 815cd15e8c5a8c4ed93f6397f2ae9266f599cf28a36c1f0854b23661aab8712c not found: ID does not exist" Jul 12 22:49:43.350988 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.350933 2579 scope.go:117] "RemoveContainer" containerID="18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00" Jul 12 22:49:43.351158 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:49:43.351142 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00\": container with ID starting with 18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00 not found: ID does not exist" containerID="18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00" Jul 12 22:49:43.351207 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.351165 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00"} err="failed to get container status \"18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00\": rpc error: code = NotFound desc = could not find container \"18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00\": container with ID starting with 18089eaaec2147734ecab6e7a3ec48a445caa670076612736cb8152a14937e00 not found: ID does not exist" Jul 12 22:49:43.353921 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.353899 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2"] Jul 12 22:49:43.355703 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.355682 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-189e5d-openshift-default-668bf49569-tjnk2"] Jul 12 22:49:43.856999 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:43.856965 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" path="/var/lib/kubelet/pods/1304b457-a93e-48e5-9933-ffb3d4a6ac4f/volumes" Jul 12 22:49:45.745385 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:45.745347 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:46.102592 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:46.102555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:46.424541 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:46.424442 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:46.524346 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:46.524313 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:46.975414 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:46.975375 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:49:49.267475 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:49.267443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:49.365554 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:49.365515 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:49.441963 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:49.441931 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:49.544669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:49.544640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:49.929547 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:49.929449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:50.029883 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:50.029849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:49:53.745131 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:49:53.745094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:02.520931 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:02.520822 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:02.621318 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:02.621278 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:03.182818 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:03.182781 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:50:10.830531 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:10.830492 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:10.932176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:10.932135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:12.608880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:12.608843 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:17.373366 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:17.373328 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:50:41.916969 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:41.916932 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:43.030572 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:43.030524 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:47.885705 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:47.885665 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:50:51.982924 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:50:51.982885 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:51:03.933376 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:51:03.933347 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:51:03.935768 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:51:03.935748 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:51:06.786077 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:51:06.786041 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:51:25.177101 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:51:25.177005 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:51:43.683623 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:51:43.683590 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:52:01.586538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:01.586501 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:52:01.687093 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:01.687061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:52:03.177995 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:03.177950 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:52:20.581871 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:20.581833 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:52:39.983029 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:39.982993 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:52:55.374338 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:55.374224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:52:56.381304 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:56.381231 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:52:59.982393 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:52:59.982353 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:53:01.679095 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.679062 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs"] Jul 12 22:53:01.679663 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.679639 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" Jul 12 22:53:01.679722 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.679666 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" Jul 12 22:53:01.679722 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.679676 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" Jul 12 22:53:01.679722 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.679682 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" Jul 12 22:53:01.679821 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.679764 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" Jul 12 22:53:01.683053 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.683035 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.686498 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.686476 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e321a4\"/\"sub-test-model-e321a4-kserve-self-signed-certs\"" Jul 12 22:53:01.686620 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.686501 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e321a4\"/\"openshift-service-ca.crt\"" Jul 12 22:53:01.686688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.686629 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e321a4\"/\"kube-root-ca.crt\"" Jul 12 22:53:01.686688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.686665 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e321a4\"/\"default-dockercfg-fhs2q\"" Jul 12 22:53:01.692425 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.692402 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs"] Jul 12 22:53:01.699316 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.699290 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:53:01.772317 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.772276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b523a7-ff6a-422f-b61f-25292cdbd2be-tls-certs\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.772509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.772365 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-tmp-dir\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.772509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.772464 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-model-cache\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.772509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.772487 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-home\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.772509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.772503 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-dshm\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.772643 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.772538 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlkls\" (UniqueName: \"kubernetes.io/projected/92b523a7-ff6a-422f-b61f-25292cdbd2be-kube-api-access-rlkls\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.873572 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.873531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-tmp-dir\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.873791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.873654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-model-cache\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.873791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.873681 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-home\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.873791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.873704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-dshm\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.873791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.873745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlkls\" (UniqueName: \"kubernetes.io/projected/92b523a7-ff6a-422f-b61f-25292cdbd2be-kube-api-access-rlkls\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.873791 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.873782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b523a7-ff6a-422f-b61f-25292cdbd2be-tls-certs\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.874059 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.874005 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-tmp-dir\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.874119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.874051 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-home\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.874119 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.874079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-model-cache\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.875993 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.875969 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-dshm\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.876380 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.876361 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b523a7-ff6a-422f-b61f-25292cdbd2be-tls-certs\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.881489 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.881465 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlkls\" (UniqueName: \"kubernetes.io/projected/92b523a7-ff6a-422f-b61f-25292cdbd2be-kube-api-access-rlkls\") pod \"sub-test-model-e321a4-kserve-8594887f59-lb2bs\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:01.993452 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:01.993352 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:02.130384 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:02.130361 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs"] Jul 12 22:53:02.132442 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:53:02.132417 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92b523a7_ff6a_422f_b61f_25292cdbd2be.slice/crio-7f2029f8a3b8f63f896333a315aea68b686b5b6aad1850e4f27350e51e4639da WatchSource:0}: Error finding container 7f2029f8a3b8f63f896333a315aea68b686b5b6aad1850e4f27350e51e4639da: Status 404 returned error can't find the container with id 7f2029f8a3b8f63f896333a315aea68b686b5b6aad1850e4f27350e51e4639da Jul 12 22:53:02.322076 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:02.322042 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" event={"ID":"92b523a7-ff6a-422f-b61f-25292cdbd2be","Type":"ContainerStarted","Data":"d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d"} Jul 12 22:53:02.322278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:02.322089 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" event={"ID":"92b523a7-ff6a-422f-b61f-25292cdbd2be","Type":"ContainerStarted","Data":"7f2029f8a3b8f63f896333a315aea68b686b5b6aad1850e4f27350e51e4639da"} Jul 12 22:53:02.338821 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:02.338766 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" podStartSLOduration=1.338750688 podStartE2EDuration="1.338750688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:53:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:53:02.337195017 +0000 UTC m=+2519.091448273" watchObservedRunningTime="2026-07-12 22:53:02.338750688 +0000 UTC m=+2519.093003944" Jul 12 22:53:11.994506 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:11.994455 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:11.994506 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:11.994523 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:12.007375 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:12.007346 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:12.384565 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:12.384531 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:53:17.681714 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:17.681675 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:53:34.510450 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:34.510413 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:53:39.280538 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:39.280501 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:53:54.690034 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:54.689997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:53:54.790860 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:53:54.790822 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:54:05.981505 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:05.981470 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:54:25.877330 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:25.877230 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:54:30.956884 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:30.956847 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs"] Jul 12 22:54:30.973931 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:30.973897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:54:31.981067 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:31.981038 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:54:43.976540 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:43.976493 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:54:45.979518 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:45.979453 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" podUID="92b523a7-ff6a-422f-b61f-25292cdbd2be" containerName="main" containerID="cri-o://d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d" gracePeriod=45 Jul 12 22:54:46.237808 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.237739 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:54:46.307218 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307184 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-home\") pod \"92b523a7-ff6a-422f-b61f-25292cdbd2be\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " Jul 12 22:54:46.307429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307295 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-tmp-dir\") pod \"92b523a7-ff6a-422f-b61f-25292cdbd2be\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " Jul 12 22:54:46.307429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307336 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-model-cache\") pod \"92b523a7-ff6a-422f-b61f-25292cdbd2be\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " Jul 12 22:54:46.307429 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307363 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b523a7-ff6a-422f-b61f-25292cdbd2be-tls-certs\") pod \"92b523a7-ff6a-422f-b61f-25292cdbd2be\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " Jul 12 22:54:46.307597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307457 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-home" (OuterVolumeSpecName: "home") pod "92b523a7-ff6a-422f-b61f-25292cdbd2be" (UID: "92b523a7-ff6a-422f-b61f-25292cdbd2be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:54:46.307597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307470 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlkls\" (UniqueName: \"kubernetes.io/projected/92b523a7-ff6a-422f-b61f-25292cdbd2be-kube-api-access-rlkls\") pod \"92b523a7-ff6a-422f-b61f-25292cdbd2be\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " Jul 12 22:54:46.307597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307540 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-dshm\") pod \"92b523a7-ff6a-422f-b61f-25292cdbd2be\" (UID: \"92b523a7-ff6a-422f-b61f-25292cdbd2be\") " Jul 12 22:54:46.307597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307554 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-model-cache" (OuterVolumeSpecName: "model-cache") pod "92b523a7-ff6a-422f-b61f-25292cdbd2be" (UID: "92b523a7-ff6a-422f-b61f-25292cdbd2be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:54:46.307597 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307572 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92b523a7-ff6a-422f-b61f-25292cdbd2be" (UID: "92b523a7-ff6a-422f-b61f-25292cdbd2be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:54:46.307906 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307883 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-home\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:54:46.308043 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307924 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-tmp-dir\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:54:46.308043 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.307935 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-model-cache\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:54:46.309653 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.309620 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-dshm" (OuterVolumeSpecName: "dshm") pod "92b523a7-ff6a-422f-b61f-25292cdbd2be" (UID: "92b523a7-ff6a-422f-b61f-25292cdbd2be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:54:46.309741 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.309678 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b523a7-ff6a-422f-b61f-25292cdbd2be-kube-api-access-rlkls" (OuterVolumeSpecName: "kube-api-access-rlkls") pod "92b523a7-ff6a-422f-b61f-25292cdbd2be" (UID: "92b523a7-ff6a-422f-b61f-25292cdbd2be"). InnerVolumeSpecName "kube-api-access-rlkls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:54:46.309741 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.309681 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b523a7-ff6a-422f-b61f-25292cdbd2be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92b523a7-ff6a-422f-b61f-25292cdbd2be" (UID: "92b523a7-ff6a-422f-b61f-25292cdbd2be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:54:46.408604 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.408570 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b523a7-ff6a-422f-b61f-25292cdbd2be-tls-certs\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:54:46.408604 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.408599 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlkls\" (UniqueName: \"kubernetes.io/projected/92b523a7-ff6a-422f-b61f-25292cdbd2be-kube-api-access-rlkls\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:54:46.408604 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.408610 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92b523a7-ff6a-422f-b61f-25292cdbd2be-dshm\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:54:46.840424 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.840389 2579 generic.go:356] "Generic (PLEG): container finished" podID="92b523a7-ff6a-422f-b61f-25292cdbd2be" containerID="d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d" exitCode=0 Jul 12 22:54:46.840611 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.840431 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" Jul 12 22:54:46.840611 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.840484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" event={"ID":"92b523a7-ff6a-422f-b61f-25292cdbd2be","Type":"ContainerDied","Data":"d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d"} Jul 12 22:54:46.840611 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.840527 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs" event={"ID":"92b523a7-ff6a-422f-b61f-25292cdbd2be","Type":"ContainerDied","Data":"7f2029f8a3b8f63f896333a315aea68b686b5b6aad1850e4f27350e51e4639da"} Jul 12 22:54:46.840611 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.840548 2579 scope.go:117] "RemoveContainer" containerID="d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d" Jul 12 22:54:46.850155 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.850140 2579 scope.go:117] "RemoveContainer" containerID="d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d" Jul 12 22:54:46.850418 ip-10-0-141-188 kubenswrapper[2579]: E0712 22:54:46.850398 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d\": container with ID starting with d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d not found: ID does not exist" containerID="d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d" Jul 12 22:54:46.850490 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.850427 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d"} err="failed to get container status \"d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d\": rpc error: code = NotFound desc = could not find container \"d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d\": container with ID starting with d7fd774cb277a98f0cea273d88addb464021a8d48cfe190f70c98581a32d591d not found: ID does not exist" Jul 12 22:54:46.861325 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.861298 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs"] Jul 12 22:54:46.864837 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:46.864815 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e321a4/sub-test-model-e321a4-kserve-8594887f59-lb2bs"] Jul 12 22:54:47.857183 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:54:47.857139 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92b523a7-ff6a-422f-b61f-25292cdbd2be" path="/var/lib/kubelet/pods/92b523a7-ff6a-422f-b61f-25292cdbd2be/volumes" Jul 12 22:55:00.811223 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:00.811187 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:55:03.439574 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:03.439542 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:55:03.558105 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:03.558061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:55:03.587435 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:03.587398 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:55:11.555666 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:11.555625 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:55:11.655744 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:11.655715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:55:23.302157 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:23.302123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:55:28.469058 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:28.469018 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:55:58.675582 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:55:58.675500 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:56:03.964664 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:03.964634 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:56:03.968106 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:03.968086 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:56:13.687964 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:13.687924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:14.778689 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:14.778644 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:56:21.020647 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:21.020614 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:22.083461 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:22.083425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:29.580106 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:29.580065 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:56:30.360450 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:30.360416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:40.770126 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.770089 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp"] Jul 12 22:56:40.770521 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.770477 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b523a7-ff6a-422f-b61f-25292cdbd2be" containerName="main" Jul 12 22:56:40.770521 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.770488 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b523a7-ff6a-422f-b61f-25292cdbd2be" containerName="main" Jul 12 22:56:40.770595 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.770562 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b523a7-ff6a-422f-b61f-25292cdbd2be" containerName="main" Jul 12 22:56:40.770595 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.770572 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1304b457-a93e-48e5-9933-ffb3d4a6ac4f" containerName="istio-proxy" Jul 12 22:56:40.773792 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.773776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:40.776149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.776116 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xlrdf\"" Jul 12 22:56:40.776149 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.776119 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 22:56:40.779687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.779666 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp"] Jul 12 22:56:40.944135 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:40.944088 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e437a13-12f2-47e1-ae4c-13a5b03f670b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp\" (UID: \"4e437a13-12f2-47e1-ae4c-13a5b03f670b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:41.045369 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.045325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e437a13-12f2-47e1-ae4c-13a5b03f670b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp\" (UID: \"4e437a13-12f2-47e1-ae4c-13a5b03f670b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:41.045981 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.045960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e437a13-12f2-47e1-ae4c-13a5b03f670b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp\" (UID: \"4e437a13-12f2-47e1-ae4c-13a5b03f670b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:41.086412 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.086381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:41.217152 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.217129 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp"] Jul 12 22:56:41.219377 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:56:41.219353 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e437a13_12f2_47e1_ae4c_13a5b03f670b.slice/crio-9a939e2444229023534985cf81352f0ac7ef1bc5fb8a7b33289b579a107883eb WatchSource:0}: Error finding container 9a939e2444229023534985cf81352f0ac7ef1bc5fb8a7b33289b579a107883eb: Status 404 returned error can't find the container with id 9a939e2444229023534985cf81352f0ac7ef1bc5fb8a7b33289b579a107883eb Jul 12 22:56:41.243408 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.243381 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:41.346453 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.346370 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:41.422736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:41.422691 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" event={"ID":"4e437a13-12f2-47e1-ae4c-13a5b03f670b","Type":"ContainerStarted","Data":"9a939e2444229023534985cf81352f0ac7ef1bc5fb8a7b33289b579a107883eb"} Jul 12 22:56:42.434572 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:42.434537 2579 generic.go:356] "Generic (PLEG): container finished" podID="4e437a13-12f2-47e1-ae4c-13a5b03f670b" containerID="1e784352ef5ccba82151c3d91784dbb92680ca75762ee8699e0fae24a7487620" exitCode=0 Jul 12 22:56:42.434955 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:42.434623 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" event={"ID":"4e437a13-12f2-47e1-ae4c-13a5b03f670b","Type":"ContainerDied","Data":"1e784352ef5ccba82151c3d91784dbb92680ca75762ee8699e0fae24a7487620"} Jul 12 22:56:43.569511 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:43.569486 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:43.669407 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:43.669371 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e437a13-12f2-47e1-ae4c-13a5b03f670b-maas-api-service-ca\") pod \"4e437a13-12f2-47e1-ae4c-13a5b03f670b\" (UID: \"4e437a13-12f2-47e1-ae4c-13a5b03f670b\") " Jul 12 22:56:43.669776 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:43.669752 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e437a13-12f2-47e1-ae4c-13a5b03f670b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e437a13-12f2-47e1-ae4c-13a5b03f670b" (UID: "4e437a13-12f2-47e1-ae4c-13a5b03f670b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:56:43.771128 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:43.771033 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e437a13-12f2-47e1-ae4c-13a5b03f670b-maas-api-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:56:44.445669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:44.445641 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" Jul 12 22:56:44.445669 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:44.445651 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp" event={"ID":"4e437a13-12f2-47e1-ae4c-13a5b03f670b","Type":"ContainerDied","Data":"9a939e2444229023534985cf81352f0ac7ef1bc5fb8a7b33289b579a107883eb"} Jul 12 22:56:44.445877 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:44.445722 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a939e2444229023534985cf81352f0ac7ef1bc5fb8a7b33289b579a107883eb" Jul 12 22:56:51.294454 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:51.294420 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:56:54.472682 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:54.472644 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:56:55.873475 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:56:55.873439 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:01.708541 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:01.708505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:01.772232 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:01.772180 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:06.545772 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.545733 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:06.786137 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.786100 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:06.892298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.892190 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s"] Jul 12 22:57:06.892607 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.892594 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e437a13-12f2-47e1-ae4c-13a5b03f670b" containerName="revoke-keys" Jul 12 22:57:06.892652 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.892608 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e437a13-12f2-47e1-ae4c-13a5b03f670b" containerName="revoke-keys" Jul 12 22:57:06.892693 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.892685 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e437a13-12f2-47e1-ae4c-13a5b03f670b" containerName="revoke-keys" Jul 12 22:57:06.895864 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.895846 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:06.898355 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.898314 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xlrdf\"" Jul 12 22:57:06.898519 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.898360 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 22:57:06.902044 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:06.902013 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s"] Jul 12 22:57:07.000224 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.000182 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9b4238-4d83-4705-8f22-3b7b4ba0f633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s\" (UID: \"2c9b4238-4d83-4705-8f22-3b7b4ba0f633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:07.101761 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.101731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9b4238-4d83-4705-8f22-3b7b4ba0f633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s\" (UID: \"2c9b4238-4d83-4705-8f22-3b7b4ba0f633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:07.102391 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.102371 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9b4238-4d83-4705-8f22-3b7b4ba0f633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s\" (UID: \"2c9b4238-4d83-4705-8f22-3b7b4ba0f633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:07.208397 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.208311 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:07.343341 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.343314 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s"] Jul 12 22:57:07.345199 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:57:07.345170 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9b4238_4d83_4705_8f22_3b7b4ba0f633.slice/crio-c70dd168a8968986a5c31742c2dd6613954b840eb72d132c69a0a4ba4c2bb6fe WatchSource:0}: Error finding container c70dd168a8968986a5c31742c2dd6613954b840eb72d132c69a0a4ba4c2bb6fe: Status 404 returned error can't find the container with id c70dd168a8968986a5c31742c2dd6613954b840eb72d132c69a0a4ba4c2bb6fe Jul 12 22:57:07.388078 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.388043 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:07.488180 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.488086 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:07.565612 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.565577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" event={"ID":"2c9b4238-4d83-4705-8f22-3b7b4ba0f633","Type":"ContainerStarted","Data":"ea314f304601b5f34523e8d48b134978875a8366580747cc5148bb17864331b4"} Jul 12 22:57:07.565987 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.565638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" event={"ID":"2c9b4238-4d83-4705-8f22-3b7b4ba0f633","Type":"ContainerStarted","Data":"c70dd168a8968986a5c31742c2dd6613954b840eb72d132c69a0a4ba4c2bb6fe"} Jul 12 22:57:07.582205 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:07.582156 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" podStartSLOduration=1.582139532 podStartE2EDuration="1.582139532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:57:07.580276778 +0000 UTC m=+2764.334530036" watchObservedRunningTime="2026-07-12 22:57:07.582139532 +0000 UTC m=+2764.336392788" Jul 12 22:57:08.571634 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:08.571597 2579 generic.go:356] "Generic (PLEG): container finished" podID="2c9b4238-4d83-4705-8f22-3b7b4ba0f633" containerID="ea314f304601b5f34523e8d48b134978875a8366580747cc5148bb17864331b4" exitCode=0 Jul 12 22:57:08.572038 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:08.571686 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" event={"ID":"2c9b4238-4d83-4705-8f22-3b7b4ba0f633","Type":"ContainerDied","Data":"ea314f304601b5f34523e8d48b134978875a8366580747cc5148bb17864331b4"} Jul 12 22:57:09.703461 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:09.703431 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:09.828553 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:09.828457 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9b4238-4d83-4705-8f22-3b7b4ba0f633-maas-api-service-ca\") pod \"2c9b4238-4d83-4705-8f22-3b7b4ba0f633\" (UID: \"2c9b4238-4d83-4705-8f22-3b7b4ba0f633\") " Jul 12 22:57:09.828803 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:09.828779 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c9b4238-4d83-4705-8f22-3b7b4ba0f633-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c9b4238-4d83-4705-8f22-3b7b4ba0f633" (UID: "2c9b4238-4d83-4705-8f22-3b7b4ba0f633"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:57:09.929703 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:09.929669 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9b4238-4d83-4705-8f22-3b7b4ba0f633-maas-api-service-ca\") on node \"ip-10-0-141-188.ec2.internal\" DevicePath \"\"" Jul 12 22:57:10.583204 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:10.583167 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" Jul 12 22:57:10.583386 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:10.583201 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s" event={"ID":"2c9b4238-4d83-4705-8f22-3b7b4ba0f633","Type":"ContainerDied","Data":"c70dd168a8968986a5c31742c2dd6613954b840eb72d132c69a0a4ba4c2bb6fe"} Jul 12 22:57:10.583386 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:10.583267 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c70dd168a8968986a5c31742c2dd6613954b840eb72d132c69a0a4ba4c2bb6fe" Jul 12 22:57:17.394702 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:17.394660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:20.579111 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:20.579070 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xp47p"] Jul 12 22:57:21.972778 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:21.972741 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:27.629579 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:27.629540 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:27.730016 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:27.729982 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:32.633988 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:32.633951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:32.857506 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:32.857468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:38.822893 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:38.822832 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-rrxr7_74b0f45b-e197-4494-ac9e-ec820d7bce7e/copy-manifests/0.log" Jul 12 22:57:38.828853 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:38.828827 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-rrxr7_74b0f45b-e197-4494-ac9e-ec820d7bce7e/manager/0.log" Jul 12 22:57:40.052756 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:40.052724 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-30d729-ktqkd_6250ca57-158c-44d1-8add-7ef1a206a69e/revoke-keys/0.log" Jul 12 22:57:40.162937 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:40.162910 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-58d7a8-9t2bk_66847cd2-6fc7-40fd-9c71-ba73a7266e36/revoke-keys/0.log" Jul 12 22:57:40.588717 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:40.588683 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-e321a4-cqxfp_4e437a13-12f2-47e1-ae4c-13a5b03f670b/revoke-keys/0.log" Jul 12 22:57:40.698463 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:40.698435 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-ecdd13-4dk4s_2c9b4238-4d83-4705-8f22-3b7b4ba0f633/revoke-keys/0.log" Jul 12 22:57:42.840540 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:42.840509 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xp47p_b137d60a-f612-47b2-80b3-caf73c46e6f9/limitador/0.log" Jul 12 22:57:43.295273 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:43.295225 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pbpsq_a83df2e9-3e4f-433a-81b5-a7b5241087b9/istio-proxy/0.log" Jul 12 22:57:43.409509 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:43.409474 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l4pbh_7919b193-f295-488e-8cb3-38ba55a3dbd0/discovery/0.log" Jul 12 22:57:43.748777 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:43.748683 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/1.log" Jul 12 22:57:44.887512 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:44.887479 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-kd8rs_39ade778-943b-4503-b748-baa90d6a19c4/main/0.log" Jul 12 22:57:49.425365 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:49.425334 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:49.521402 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:49.521351 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" limit=1024 Jul 12 22:57:52.251910 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:52.251872 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nj7st_1a2ca8a7-f8d9-4fc1-a5ea-f9e29c87dd8a/global-pull-secret-syncer/0.log" Jul 12 22:57:52.300198 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:52.300162 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dd2q2_165d9e3d-8328-4146-b17a-c3b9348a411c/konnectivity-agent/0.log" Jul 12 22:57:52.437586 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:52.437551 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-188.ec2.internal_81f992adf125e091bbf7fa1071ef9694/haproxy/0.log" Jul 12 22:57:57.576278 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:57.576225 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xp47p_b137d60a-f612-47b2-80b3-caf73c46e6f9/limitador/0.log" Jul 12 22:57:59.122300 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.122274 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/alertmanager/0.log" Jul 12 22:57:59.144804 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.144780 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/config-reloader/0.log" Jul 12 22:57:59.164785 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.164761 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/kube-rbac-proxy-web/0.log" Jul 12 22:57:59.187005 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.186978 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/kube-rbac-proxy/0.log" Jul 12 22:57:59.209508 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.209485 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/kube-rbac-proxy-metric/0.log" Jul 12 22:57:59.229360 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.229339 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/prom-label-proxy/0.log" Jul 12 22:57:59.251812 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.251785 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bbc1e1c4-0292-48b5-bc12-3d61242d4f15/init-config-reloader/0.log" Jul 12 22:57:59.319413 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.319381 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-slkwk_708dc70e-0fea-42bd-8ec9-1ae79b77f4fb/kube-state-metrics/0.log" Jul 12 22:57:59.341254 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.341213 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-slkwk_708dc70e-0fea-42bd-8ec9-1ae79b77f4fb/kube-rbac-proxy-main/0.log" Jul 12 22:57:59.362617 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.362593 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-slkwk_708dc70e-0fea-42bd-8ec9-1ae79b77f4fb/kube-rbac-proxy-self/0.log" Jul 12 22:57:59.395046 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.395015 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6bb9d9d469-sjdnd_4584f9f2-7823-45a5-821c-b27bd66de9d1/metrics-server/0.log" Jul 12 22:57:59.424537 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.424512 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-q5jc5_777e8538-08a1-4a20-8f07-bc9ed5fc28c7/monitoring-plugin/0.log" Jul 12 22:57:59.533673 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.533648 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8dht_c10680b7-41b7-4323-90cd-3e69a3f9b1c8/node-exporter/0.log" Jul 12 22:57:59.553302 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.553214 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8dht_c10680b7-41b7-4323-90cd-3e69a3f9b1c8/kube-rbac-proxy/0.log" Jul 12 22:57:59.574373 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.574339 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8dht_c10680b7-41b7-4323-90cd-3e69a3f9b1c8/init-textfile/0.log" Jul 12 22:57:59.679790 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.679763 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-jx2rn_f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95/kube-rbac-proxy-main/0.log" Jul 12 22:57:59.705151 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.705121 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-jx2rn_f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95/kube-rbac-proxy-self/0.log" Jul 12 22:57:59.724894 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.724870 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-jx2rn_f2ed0ad5-c3e4-4c61-a59f-7bd0d4a3ca95/openshift-state-metrics/0.log" Jul 12 22:57:59.772507 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.772481 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/prometheus/0.log" Jul 12 22:57:59.792660 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.792632 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/config-reloader/0.log" Jul 12 22:57:59.813658 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.813577 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/thanos-sidecar/0.log" Jul 12 22:57:59.837252 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.837226 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/kube-rbac-proxy-web/0.log" Jul 12 22:57:59.862332 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.862305 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/kube-rbac-proxy/0.log" Jul 12 22:57:59.887552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.887527 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/kube-rbac-proxy-thanos/0.log" Jul 12 22:57:59.911342 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.911320 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_81202aa5-f465-4c08-ab9d-1153df129686/init-config-reloader/0.log" Jul 12 22:57:59.943063 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.943034 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lxt58_db3db6c9-e7b1-4099-bbb9-797b0ef02b21/prometheus-operator/0.log" Jul 12 22:57:59.964359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:57:59.964335 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lxt58_db3db6c9-e7b1-4099-bbb9-797b0ef02b21/kube-rbac-proxy/0.log" Jul 12 22:58:00.011298 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.011271 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-q5vgs_e54d14e9-eb17-4ff8-837e-ce6b8580b44a/prometheus-operator-admission-webhook/0.log" Jul 12 22:58:00.042668 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.042635 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9747fbc6-jz8nt_246e296d-fd4b-4ece-8773-db0d25d03299/telemeter-client/0.log" Jul 12 22:58:00.075686 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.075603 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9747fbc6-jz8nt_246e296d-fd4b-4ece-8773-db0d25d03299/reload/0.log" Jul 12 22:58:00.096532 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.096499 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9747fbc6-jz8nt_246e296d-fd4b-4ece-8773-db0d25d03299/kube-rbac-proxy/0.log" Jul 12 22:58:00.128476 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.128310 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b4ddd8b5-npbnz_7eae20f3-801a-4c00-837a-b3b62f6f1461/thanos-query/0.log" Jul 12 22:58:00.162515 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.162486 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b4ddd8b5-npbnz_7eae20f3-801a-4c00-837a-b3b62f6f1461/kube-rbac-proxy-web/0.log" Jul 12 22:58:00.186940 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.186911 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b4ddd8b5-npbnz_7eae20f3-801a-4c00-837a-b3b62f6f1461/kube-rbac-proxy/0.log" Jul 12 22:58:00.207523 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.207492 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b4ddd8b5-npbnz_7eae20f3-801a-4c00-837a-b3b62f6f1461/prom-label-proxy/0.log" Jul 12 22:58:00.228152 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.228122 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b4ddd8b5-npbnz_7eae20f3-801a-4c00-837a-b3b62f6f1461/kube-rbac-proxy-rules/0.log" Jul 12 22:58:00.252889 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:00.252856 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b4ddd8b5-npbnz_7eae20f3-801a-4c00-837a-b3b62f6f1461/kube-rbac-proxy-metrics/0.log" Jul 12 22:58:02.759291 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.759260 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-784f58b8b-hgk7h_2a245ac5-cdb6-4539-82af-5c3f84e429ec/console/0.log" Jul 12 22:58:02.987880 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.987840 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss"] Jul 12 22:58:02.988331 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.988313 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9b4238-4d83-4705-8f22-3b7b4ba0f633" containerName="revoke-keys" Jul 12 22:58:02.988331 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.988333 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9b4238-4d83-4705-8f22-3b7b4ba0f633" containerName="revoke-keys" Jul 12 22:58:02.988479 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.988434 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c9b4238-4d83-4705-8f22-3b7b4ba0f633" containerName="revoke-keys" Jul 12 22:58:02.991767 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.991746 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:02.994133 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.994115 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkcz8\"/\"kube-root-ca.crt\"" Jul 12 22:58:02.994225 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.994119 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xkcz8\"/\"default-dockercfg-kqqvs\"" Jul 12 22:58:02.995736 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.995717 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkcz8\"/\"openshift-service-ca.crt\"" Jul 12 22:58:02.997933 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:02.997911 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss"] Jul 12 22:58:03.052820 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.052784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2dbw\" (UniqueName: \"kubernetes.io/projected/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-kube-api-access-m2dbw\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.052820 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.052825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-podres\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.053043 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.052853 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-lib-modules\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.053043 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.052914 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-proc\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.053043 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.052962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-sys\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153637 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2dbw\" (UniqueName: \"kubernetes.io/projected/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-kube-api-access-m2dbw\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153687 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-podres\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153729 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-lib-modules\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-proc\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-sys\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153885 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-podres\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153890 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-lib-modules\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.153949 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153919 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-proc\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.154176 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.153984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-sys\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.161154 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.161127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2dbw\" (UniqueName: \"kubernetes.io/projected/6bcdbef2-3ba9-46b6-aa84-c9afd998ca15-kube-api-access-m2dbw\") pod \"perf-node-gather-daemonset-9hnss\" (UID: \"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.303369 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.303275 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.437743 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.437719 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss"] Jul 12 22:58:03.439724 ip-10-0-141-188 kubenswrapper[2579]: W0712 22:58:03.439689 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6bcdbef2_3ba9_46b6_aa84_c9afd998ca15.slice/crio-25e1c435b30e9b4e307388996096eeb942e4351ba0709f4c9b958aa21f0c3a83 WatchSource:0}: Error finding container 25e1c435b30e9b4e307388996096eeb942e4351ba0709f4c9b958aa21f0c3a83: Status 404 returned error can't find the container with id 25e1c435b30e9b4e307388996096eeb942e4351ba0709f4c9b958aa21f0c3a83 Jul 12 22:58:03.847749 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.847711 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" event={"ID":"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15","Type":"ContainerStarted","Data":"ee279eb85ad708f66ecf40da00d4e46965ca029c0b8da963ca3d69ee6bb7ba1e"} Jul 12 22:58:03.848232 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.847765 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" event={"ID":"6bcdbef2-3ba9-46b6-aa84-c9afd998ca15","Type":"ContainerStarted","Data":"25e1c435b30e9b4e307388996096eeb942e4351ba0709f4c9b958aa21f0c3a83"} Jul 12 22:58:03.848232 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.847797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:03.862830 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:03.862779 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" podStartSLOduration=1.8627660069999998 podStartE2EDuration="1.862766007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:58:03.861572134 +0000 UTC m=+2820.615825393" watchObservedRunningTime="2026-07-12 22:58:03.862766007 +0000 UTC m=+2820.617019263" Jul 12 22:58:04.472200 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:04.472167 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zrn4_47713c46-5f1b-4b28-be04-9d7abbddb8ee/dns/0.log" Jul 12 22:58:04.491684 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:04.491649 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zrn4_47713c46-5f1b-4b28-be04-9d7abbddb8ee/kube-rbac-proxy/0.log" Jul 12 22:58:04.620350 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:04.620323 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jtvzw_6750b24d-b4dc-4842-b0a8-91d90c255345/dns-node-resolver/0.log" Jul 12 22:58:05.193131 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:05.193106 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tbnjt_7a5ffff7-47c3-4aad-8d8a-9be8675d178e/node-ca/0.log" Jul 12 22:58:06.104709 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:06.104678 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pbpsq_a83df2e9-3e4f-433a-81b5-a7b5241087b9/istio-proxy/0.log" Jul 12 22:58:06.220129 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:06.220099 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l4pbh_7919b193-f295-488e-8cb3-38ba55a3dbd0/discovery/0.log" Jul 12 22:58:06.291876 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:06.291847 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/0.log" Jul 12 22:58:06.323359 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:06.323326 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/1.log" Jul 12 22:58:07.120090 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:07.120061 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4f6cd_a5769b66-5ecb-4684-95f3-bb079fea4583/serve-healthcheck-canary/0.log" Jul 12 22:58:07.664577 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:07.664550 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ckzz_d4b6e816-edc3-4854-b822-446e867f39c7/kube-rbac-proxy/0.log" Jul 12 22:58:07.685963 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:07.685933 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ckzz_d4b6e816-edc3-4854-b822-446e867f39c7/exporter/0.log" Jul 12 22:58:07.705779 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:07.705756 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ckzz_d4b6e816-edc3-4854-b822-446e867f39c7/extractor/0.log" Jul 12 22:58:09.864027 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:09.863991 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-9hnss" Jul 12 22:58:10.329175 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:10.329143 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-rrxr7_74b0f45b-e197-4494-ac9e-ec820d7bce7e/manager/0.log" Jul 12 22:58:10.347788 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:10.347761 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-rrxr7_74b0f45b-e197-4494-ac9e-ec820d7bce7e/copy-manifests/0.log" Jul 12 22:58:11.546688 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:11.546665 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64b6b6c6c7-jc64b_ac51e41a-aa72-4ef7-b90a-8db803b941c0/manager/0.log" Jul 12 22:58:11.597106 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:11.597073 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rms7d_c578efd2-166c-4fc7-b298-0e50c2345358/openshift-lws-operator/0.log" Jul 12 22:58:13.153459 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:13.153431 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pbpsq_a83df2e9-3e4f-433a-81b5-a7b5241087b9/istio-proxy/0.log" Jul 12 22:58:13.267102 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:13.267008 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l4pbh_7919b193-f295-488e-8cb3-38ba55a3dbd0/discovery/0.log" Jul 12 22:58:13.310752 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:13.310716 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/0.log" Jul 12 22:58:13.340691 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:13.340658 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/1.log" Jul 12 22:58:14.659667 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:14.659643 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pbpsq_a83df2e9-3e4f-433a-81b5-a7b5241087b9/istio-proxy/0.log" Jul 12 22:58:14.779592 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:14.779563 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l4pbh_7919b193-f295-488e-8cb3-38ba55a3dbd0/discovery/0.log" Jul 12 22:58:14.830222 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:14.830195 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/0.log" Jul 12 22:58:14.861800 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:14.861766 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2dtxr_54e17f15-1e28-4280-b369-fb272283881d/istio-proxy/1.log" Jul 12 22:58:18.250917 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.250882 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/kube-multus-additional-cni-plugins/0.log" Jul 12 22:58:18.270076 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.270049 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/egress-router-binary-copy/0.log" Jul 12 22:58:18.290407 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.290375 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/cni-plugins/0.log" Jul 12 22:58:18.308670 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.308645 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/bond-cni-plugin/0.log" Jul 12 22:58:18.328070 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.328040 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/routeoverride-cni/0.log" Jul 12 22:58:18.346486 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.346460 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/whereabouts-cni-bincopy/0.log" Jul 12 22:58:18.365846 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.365818 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fbdv_1cfdeee8-d3fc-4789-996f-0db8e40a4064/whereabouts-cni/0.log" Jul 12 22:58:18.722339 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.722309 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rtcqm_d4a3127f-38ce-4b6d-a9a0-d4c0ec1830da/kube-multus/0.log" Jul 12 22:58:18.740552 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.740526 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7c5q2_58ae1d9a-b83c-4fa0-adb6-385298d8c5f7/network-metrics-daemon/0.log" Jul 12 22:58:18.758645 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:18.758619 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7c5q2_58ae1d9a-b83c-4fa0-adb6-385298d8c5f7/kube-rbac-proxy/0.log" Jul 12 22:58:20.075034 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.075006 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-controller/0.log" Jul 12 22:58:20.094408 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.094377 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/1.log" Jul 12 22:58:20.094562 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.094418 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovn-acl-logging/0.log" Jul 12 22:58:20.109754 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.109733 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/kube-rbac-proxy-node/0.log" Jul 12 22:58:20.128957 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.128933 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 22:58:20.148548 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.148524 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/northd/0.log" Jul 12 22:58:20.169100 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.169074 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/nbdb/0.log" Jul 12 22:58:20.188624 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.188603 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/sbdb/0.log" Jul 12 22:58:20.240659 ip-10-0-141-188 kubenswrapper[2579]: I0712 22:58:20.240625 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n9zg9_edbb9cc6-2861-494e-b593-f8a0cad404ea/ovnkube-controller/0.log"