Jul 23 18:50:47.304108 ip-10-0-141-105 systemd[1]: Starting Kubernetes Kubelet... Jul 23 18:50:47.682757 ip-10-0-141-105 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 18:50:47.682757 ip-10-0-141-105 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 18:50:47.682757 ip-10-0-141-105 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 18:50:47.682757 ip-10-0-141-105 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 18:50:47.682757 ip-10-0-141-105 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 18:50:47.683466 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.682837 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 18:50:47.687400 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687384 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:50:47.687400 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687401 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687407 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687410 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687413 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687416 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687418 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687421 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687424 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687426 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687429 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687432 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687434 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687437 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687446 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687449 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687452 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687455 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687457 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687460 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687462 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:50:47.687464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687464 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687467 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687470 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687472 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687475 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687478 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687481 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687485 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687488 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687491 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687493 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687496 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687500 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687503 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687507 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687509 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687512 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687515 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687517 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:50:47.687960 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687520 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687523 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687525 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687528 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687531 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687533 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687535 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687538 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687540 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687543 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687545 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687548 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687550 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687552 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687554 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687558 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687560 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687563 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687566 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687568 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:50:47.688411 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687571 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687573 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687576 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687578 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687581 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687585 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687589 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687591 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687594 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687596 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687599 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687601 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687606 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687608 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687611 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687613 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687616 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687618 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687621 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687623 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:50:47.689015 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687626 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687628 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687630 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687633 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687635 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687638 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687640 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687644 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687648 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687650 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687652 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687655 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687657 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687660 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687662 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687665 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687667 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687670 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.687673 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688443 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:50:47.689501 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688458 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688462 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688465 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688468 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688471 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688474 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688477 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688479 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688481 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688484 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688486 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688489 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688491 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688494 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688496 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688500 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688502 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688504 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688507 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688510 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:50:47.689998 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688513 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688515 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688518 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688520 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688523 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688525 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688528 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688531 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688533 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688536 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688538 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688540 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688544 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688548 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688550 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688553 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688556 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688558 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688561 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:50:47.690528 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688564 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688568 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688572 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688576 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688581 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688584 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688586 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688589 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688592 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688595 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688598 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688601 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688604 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688607 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688610 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688612 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688615 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688617 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688620 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:50:47.691008 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688622 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688625 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688627 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688630 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688632 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688635 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688638 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688642 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688645 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688648 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688650 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688653 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688655 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688658 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688660 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688663 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688665 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688667 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688670 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688672 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:50:47.691475 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688675 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688677 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688680 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688682 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688685 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688687 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688689 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688692 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688694 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688697 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688701 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688704 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688707 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688710 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688713 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688715 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688718 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688721 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688725 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:50:47.691985 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.688727 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689131 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689137 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689141 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689144 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689147 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689150 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689153 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689156 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689160 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689162 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689165 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689168 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689171 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689173 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689176 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689179 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689181 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:50:47.692443 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689184 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689188 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689192 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689194 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689197 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689200 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689202 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689205 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689207 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689209 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689212 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689215 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689217 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689223 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689225 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689228 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689230 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689233 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689235 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689239 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:50:47.692913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689241 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689244 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689246 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689249 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689251 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689253 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689256 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689258 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689261 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689263 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689273 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689276 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689278 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689281 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689284 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689286 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689289 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689291 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689294 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689297 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:50:47.693402 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689299 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689302 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689304 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689307 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689310 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689313 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689316 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689319 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689322 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689325 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689328 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689330 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689333 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689335 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689338 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689340 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689343 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689345 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689348 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689350 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:50:47.693895 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689353 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689355 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689357 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689360 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689362 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689365 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689367 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689370 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689372 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689374 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689377 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689379 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689381 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689384 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689387 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689390 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689393 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689397 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689399 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689402 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:50:47.694358 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689404 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.689407 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689478 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689485 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689492 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689496 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689501 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689504 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689508 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689513 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689516 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689520 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689524 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689527 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689530 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689533 2578 flags.go:64] FLAG: --cgroup-root="" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689536 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689539 2578 flags.go:64] FLAG: --client-ca-file="" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689542 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689545 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689550 2578 flags.go:64] FLAG: --cluster-domain="" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689553 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689557 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689560 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 18:50:47.694830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689563 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689567 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689570 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689573 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689576 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689580 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689583 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689585 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689588 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689591 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689596 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689599 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689602 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689605 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689608 2578 flags.go:64] FLAG: --enable-server="true" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689611 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689616 2578 flags.go:64] FLAG: --event-burst="100" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689619 2578 flags.go:64] FLAG: --event-qps="50" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689622 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689625 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689628 2578 flags.go:64] FLAG: --eviction-hard="" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689632 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689635 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689638 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689640 2578 flags.go:64] FLAG: --eviction-soft="" Jul 23 18:50:47.695431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689643 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689646 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689649 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689653 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689656 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689658 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689661 2578 flags.go:64] FLAG: --feature-gates="" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689665 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689668 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689671 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689674 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689677 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689681 2578 flags.go:64] FLAG: --help="false" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689684 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-141-105.ec2.internal" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689687 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689690 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689693 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689697 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689701 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689703 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689707 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689709 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689712 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 18:50:47.696052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689715 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689718 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689721 2578 flags.go:64] FLAG: --kube-reserved="" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689724 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689726 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689729 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689732 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689735 2578 flags.go:64] FLAG: --lock-file="" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689738 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689741 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689744 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689749 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689753 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689756 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689758 2578 flags.go:64] FLAG: --logging-format="text" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689761 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689764 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689767 2578 flags.go:64] FLAG: --manifest-url="" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689770 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689774 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689777 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689782 2578 flags.go:64] FLAG: --max-pods="110" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689785 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689788 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689791 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 18:50:47.696619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689794 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689797 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689800 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689803 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689811 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689814 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689830 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689834 2578 flags.go:64] FLAG: --pod-cidr="" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689837 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689842 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689846 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689849 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689852 2578 flags.go:64] FLAG: --port="10250" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689855 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689858 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af13043bab09a610" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689861 2578 flags.go:64] FLAG: --qos-reserved="" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689864 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689867 2578 flags.go:64] FLAG: --register-node="true" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689870 2578 flags.go:64] FLAG: --register-with-taints="" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689879 2578 flags.go:64] FLAG: --registry-burst="10" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689882 2578 flags.go:64] FLAG: --registry-qps="5" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689885 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689888 2578 flags.go:64] FLAG: --reserved-memory="" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689891 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689894 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 18:50:47.697251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689897 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689900 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689903 2578 flags.go:64] FLAG: --runonce="false" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689906 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689909 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689912 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689916 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689919 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689922 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689925 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689927 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689931 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689934 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689936 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689939 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689942 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689945 2578 flags.go:64] FLAG: --system-cgroups="" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689948 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689953 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689956 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689958 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689961 2578 flags.go:64] FLAG: --tls-min-version="" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689964 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689967 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689969 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 18:50:47.698079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689972 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689976 2578 flags.go:64] FLAG: --v="2" Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689980 2578 flags.go:64] FLAG: --version="false" Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689984 2578 flags.go:64] FLAG: --vmodule="" Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689988 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.689991 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690111 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690116 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690118 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690121 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690123 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690127 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690130 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690132 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690135 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690137 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690140 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690142 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690145 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690148 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690150 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:50:47.698950 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690153 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690155 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690158 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690160 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690163 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690165 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690168 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690170 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690173 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690175 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690178 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690180 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690184 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690186 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690189 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690191 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690194 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690196 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690199 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690201 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:50:47.699485 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690204 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690206 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690210 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690212 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690215 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690217 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690220 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690222 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690225 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690227 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690230 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690234 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690236 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690239 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690241 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690244 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690246 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690249 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690251 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690254 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:50:47.699996 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690256 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690258 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690261 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690263 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690267 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690270 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690272 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690274 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690277 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690279 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690282 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690284 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690287 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690290 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690295 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690298 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690301 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690304 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690307 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:50:47.700489 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690310 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690312 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690316 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690319 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690323 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690327 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690331 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690333 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690336 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690339 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690341 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690344 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690346 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690348 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690351 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690353 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690356 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690360 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690362 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:50:47.700972 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690365 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690367 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690370 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690372 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690375 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.690378 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.691848 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.700177 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.700197 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.701096 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.701106 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701168 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701175 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701178 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701181 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:50:47.701435 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701184 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701187 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701190 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701193 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701195 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701198 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701201 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701204 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701207 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701209 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701212 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701215 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701219 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701223 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701226 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701228 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701232 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701234 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701237 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:50:47.701800 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701239 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701242 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701245 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701248 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701250 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701254 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701258 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701261 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701264 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701273 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701277 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701280 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701283 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701285 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701288 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701291 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701293 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701296 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701299 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:50:47.702392 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701301 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701304 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701306 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701309 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701313 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701316 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701319 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701321 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701324 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701326 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701329 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701332 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701334 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701337 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701339 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701342 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701345 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701347 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701350 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:50:47.702913 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701353 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701355 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701358 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701361 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701363 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701366 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701370 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701373 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701376 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701379 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701381 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701384 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701386 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701389 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701391 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701394 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701396 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701399 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701402 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701405 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:50:47.703384 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701407 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701410 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701413 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701415 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701418 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701420 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701423 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701425 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701428 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701430 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701433 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701436 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701439 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701442 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701444 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701447 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701449 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:50:47.703890 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701452 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.701457 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701592 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701598 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701602 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701605 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701610 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701614 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701617 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701620 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701623 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701626 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701629 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701631 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:50:47.704287 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701634 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701637 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701639 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701642 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701645 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701647 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701650 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701652 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701655 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701657 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701660 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701662 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701665 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701667 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701670 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701673 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701675 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701677 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701680 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701682 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:50:47.704636 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701685 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701687 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701689 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701692 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701694 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701697 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701699 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701702 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701704 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701706 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701709 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701711 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701713 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701716 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701719 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701722 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701726 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701729 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701732 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701735 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:50:47.705147 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701737 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701740 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701742 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701745 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701747 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701750 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701752 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701755 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701758 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701760 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701763 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701765 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701768 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701770 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701773 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701775 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701778 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701780 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701783 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701785 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:50:47.705630 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701788 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701790 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701792 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701795 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701797 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701799 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701804 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701806 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701809 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701812 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701815 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701834 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701838 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701841 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701843 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701846 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701849 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701851 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701854 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:50:47.706172 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701857 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701859 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701862 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701864 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701867 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701869 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701872 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:47.701874 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.701879 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.702004 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 18:50:47.706633 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.706154 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 18:50:47.707384 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.707373 2578 server.go:1019] "Starting client certificate rotation" Jul 23 18:50:47.707493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.707474 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 18:50:47.707522 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.707515 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 18:50:47.710006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.709992 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 18:50:47.711361 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.711348 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 18:50:47.721455 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.721433 2578 log.go:25] "Validated CRI v1 runtime API" Jul 23 18:50:47.723856 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.723841 2578 log.go:25] "Validated CRI v1 image API" Jul 23 18:50:47.725251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.725226 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 18:50:47.727378 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.727360 2578 fs.go:135] Filesystem UUIDs: map[5ebda0ca-5ed6-4adf-a7ad-c0d3d715511c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 980247a1-7b4a-4d1f-a1c9-487e5fa97953:/dev/nvme0n1p3] Jul 23 18:50:47.727432 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.727378 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 18:50:47.733070 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.732966 2578 manager.go:216] Machine: {Timestamp:2026-07-23 18:50:47.7316205 +0000 UTC m=+0.312764710 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099711 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d65b0ef0ed84cb8cba08c5524c3cb SystemUUID:ec2d65b0-ef0e-d84c-b8cb-a08c5524c3cb BootID:3f28d0c3-7f3a-45ec-b21c-b403627dea18 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3a:b7:74:0b:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3a:b7:74:0b:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:cb:98:e4:4b:04 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 18:50:47.733070 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.733069 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 18:50:47.733205 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.733147 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 18:50:47.736447 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.736422 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 18:50:47.736606 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.736449 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 18:50:47.737887 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.737866 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 18:50:47.737947 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.737934 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 18:50:47.738006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.737954 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 18:50:47.738587 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.738566 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 18:50:47.739915 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.739899 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 18:50:47.742260 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.742242 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 18:50:47.742840 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.742809 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 18:50:47.742966 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.742955 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 18:50:47.743032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.742995 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 23 18:50:47.743032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.743010 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 18:50:47.743032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.743031 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 18:50:47.743154 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.743045 2578 kubelet.go:398] "Adding apiserver pod source" Jul 23 18:50:47.743154 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.743057 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 18:50:47.743457 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.743444 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 18:50:47.743534 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.743469 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 18:50:47.745616 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.745600 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 18:50:47.747022 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.747009 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 18:50:47.747061 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.747039 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 18:50:47.748159 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748147 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748164 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748171 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748176 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748185 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748190 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748198 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 18:50:47.748206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748204 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 18:50:47.748377 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748210 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 18:50:47.748377 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748216 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 18:50:47.748377 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748225 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 18:50:47.748377 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.748231 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 18:50:47.749549 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.749536 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 18:50:47.749585 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.749553 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 18:50:47.753006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.752993 2578 server.go:1268] "Started kubelet" Jul 23 18:50:47.753102 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.753078 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 18:50:47.753189 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.753144 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 18:50:47.753245 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.753212 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 18:50:47.754226 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.754206 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 18:50:47.754293 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.754273 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 18:50:47.754583 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.754567 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 18:50:47.755723 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.755704 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 18:50:47.755989 ip-10-0-141-105 systemd[1]: Started Kubernetes Kubelet. Jul 23 18:50:47.757514 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.757498 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 23 18:50:47.760465 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.760447 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 18:50:47.761256 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.760276 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-105.ec2.internal.18c4ffd630d5de6a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-105.ec2.internal,UID:ip-10-0-141-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-105.ec2.internal,},FirstTimestamp:2026-07-23 18:50:47.752965738 +0000 UTC m=+0.334109949,LastTimestamp:2026-07-23 18:50:47.752965738 +0000 UTC m=+0.334109949,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-105.ec2.internal,}" Jul 23 18:50:47.761708 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.761693 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 18:50:47.761758 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.761696 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 18:50:47.761860 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.761841 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:47.762486 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762473 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 18:50:47.762618 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762466 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 18:50:47.762618 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762611 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 18:50:47.762730 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762623 2578 factory.go:153] Registering CRI-O factory Jul 23 18:50:47.762730 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762650 2578 factory.go:223] Registration of the crio container factory successfully Jul 23 18:50:47.762730 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762711 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 23 18:50:47.762730 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762720 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 23 18:50:47.762730 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762720 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 18:50:47.762959 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762740 2578 factory.go:55] Registering systemd factory Jul 23 18:50:47.762959 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762749 2578 factory.go:223] Registration of the systemd container factory successfully Jul 23 18:50:47.762959 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762768 2578 factory.go:103] Registering Raw factory Jul 23 18:50:47.762959 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.762782 2578 manager.go:1196] Started watching for new ooms in manager Jul 23 18:50:47.763253 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.763240 2578 manager.go:349] Starting recovery of all containers Jul 23 18:50:47.764386 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.764363 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 18:50:47.764630 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.764609 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 18:50:47.772609 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.772484 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9b62b" Jul 23 18:50:47.774666 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.774647 2578 manager.go:354] Recovery completed Jul 23 18:50:47.778986 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.778973 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:47.780726 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.780710 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:47.780804 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.780737 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:47.780804 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.780748 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:47.781310 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.781296 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 18:50:47.781310 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.781309 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 18:50:47.781409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.781323 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 18:50:47.781409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.781329 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:47.781688 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.781674 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9b62b" Jul 23 18:50:47.782041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.782024 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:47.782088 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.782057 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:47.782088 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.782071 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:47.783028 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.783008 2578 policy_none.go:49] "None policy: Start" Jul 23 18:50:47.783028 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.783031 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 18:50:47.783150 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.783045 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 18:50:47.783150 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.782996 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-105.ec2.internal.18c4ffd6327d6e62 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-105.ec2.internal,UID:ip-10-0-141-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-105.ec2.internal,},FirstTimestamp:2026-07-23 18:50:47.780724322 +0000 UTC m=+0.361868535,LastTimestamp:2026-07-23 18:50:47.780724322 +0000 UTC m=+0.361868535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-105.ec2.internal,}" Jul 23 18:50:47.784097 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.784067 2578 policy_none.go:47] "Start" Jul 23 18:50:47.821653 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.821448 2578 manager.go:337] "Starting Device Plugin manager" Jul 23 18:50:47.821756 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.821678 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 18:50:47.821756 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.821691 2578 server.go:87] "Starting device plugin registration server" Jul 23 18:50:47.821983 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.821971 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 18:50:47.822035 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.822012 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 18:50:47.822155 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.822132 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 18:50:47.822316 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.822209 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 18:50:47.822316 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.822217 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 18:50:47.822777 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.822753 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 18:50:47.822869 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.822814 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:47.914945 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.914913 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 18:50:47.916282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.916263 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 18:50:47.916349 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.916297 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 18:50:47.916349 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.916321 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 18:50:47.916425 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.916366 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 18:50:47.919333 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.919316 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:50:47.922529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.922516 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:47.923312 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.923296 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:47.923388 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.923323 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:47.923388 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.923334 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:47.923452 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.923390 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-105.ec2.internal" Jul 23 18:50:47.941903 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:47.941853 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-105.ec2.internal" Jul 23 18:50:47.941903 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.941876 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-105.ec2.internal\": node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:47.973147 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:47.973121 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.016572 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.016543 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"] Jul 23 18:50:48.016690 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.016625 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:48.018109 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.018093 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:48.018193 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.018120 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:48.018193 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.018129 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:48.019690 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.019672 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:48.019862 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.019848 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.019908 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.019890 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:48.020397 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.020382 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:48.020397 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.020390 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:48.020493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.020410 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:48.020493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.020411 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:48.020493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.020421 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:48.020493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.020454 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:48.021670 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.021649 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.021724 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.021674 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:50:48.022425 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.022400 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:50:48.022533 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.022443 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:50:48.022533 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.022457 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:50:48.049627 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.049599 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-105.ec2.internal\" not found" node="ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.054160 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.054143 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-105.ec2.internal\" not found" node="ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.064382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.064366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.064444 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.064390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.064444 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.064408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d37fd728dd468b7ee1c820042e75b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-105.ec2.internal\" (UID: \"3d37fd728dd468b7ee1c820042e75b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.073782 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.073766 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.164987 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.164954 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.164987 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.164989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.165180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.165013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d37fd728dd468b7ee1c820042e75b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-105.ec2.internal\" (UID: \"3d37fd728dd468b7ee1c820042e75b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.165180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.165058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d37fd728dd468b7ee1c820042e75b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-105.ec2.internal\" (UID: \"3d37fd728dd468b7ee1c820042e75b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.165180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.165063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.165180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.165063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.174080 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.174050 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.274968 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.274898 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.352152 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.352124 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.356794 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.356779 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" Jul 23 18:50:48.375510 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.375488 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.476031 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.475994 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.576598 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.576521 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.657413 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.657386 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:50:48.677005 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.676976 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.707146 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.707121 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 18:50:48.707576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.707298 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 18:50:48.707576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.707304 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 18:50:48.777364 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.777336 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.785502 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.785473 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 18:45:48 +0000 UTC" deadline="2028-04-27 16:22:55.277150548 +0000 UTC" Jul 23 18:50:48.785502 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.785500 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15453h32m6.491654018s" Jul 23 18:50:48.878186 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.878126 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.908571 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.908543 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:50:48.979148 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:48.979122 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:48.981532 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:48.981481 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666bf80ba1e1983d2d3dd851bc8ab130.slice/crio-598a2ff15dc259a06e8d004a4d0bf16513013ad90479f176f270e07ce46d2ef4 WatchSource:0}: Error finding container 598a2ff15dc259a06e8d004a4d0bf16513013ad90479f176f270e07ce46d2ef4: Status 404 returned error can't find the container with id 598a2ff15dc259a06e8d004a4d0bf16513013ad90479f176f270e07ce46d2ef4 Jul 23 18:50:48.981743 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:48.981727 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d37fd728dd468b7ee1c820042e75b2d.slice/crio-9d5f1eb079733ff4a084810a97fc89bb736f895829735cd198c8207796b915f8 WatchSource:0}: Error finding container 9d5f1eb079733ff4a084810a97fc89bb736f895829735cd198c8207796b915f8: Status 404 returned error can't find the container with id 9d5f1eb079733ff4a084810a97fc89bb736f895829735cd198c8207796b915f8 Jul 23 18:50:48.987282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:48.987264 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:50:49.079477 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.079432 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:49.180645 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.180568 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found" Jul 23 18:50:49.248030 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.248006 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:50:49.262345 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.262305 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" Jul 23 18:50:49.277950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.277927 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 18:50:49.279025 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.279010 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" Jul 23 18:50:49.289945 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.289924 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 18:50:49.744640 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.744607 2578 apiserver.go:52] "Watching apiserver" Jul 23 18:50:49.754315 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.754288 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 18:50:49.754795 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.754767 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-wkn9h","openshift-network-operator/iptables-alerter-2fwdn","kube-system/global-pull-secret-syncer-wphxg","kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gnlxr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal","openshift-multus/multus-8gshw","openshift-multus/multus-additional-cni-plugins-7h7t6","openshift-ovn-kubernetes/ovnkube-node-6wwvr","kube-system/konnectivity-agent-4c2k5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz","openshift-dns/node-resolver-8dcd6","openshift-image-registry/node-ca-xlhsn","openshift-multus/network-metrics-daemon-zbwbh"] Jul 23 18:50:49.756699 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.756677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.759926 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.759766 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 18:50:49.760048 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.760029 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 18:50:49.760301 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.760261 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z268j\"" Jul 23 18:50:49.762224 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.762060 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 18:50:49.764738 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.764717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.766615 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.766594 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.766714 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.766667 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:50:49.768016 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.767997 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7zbfk\"" Jul 23 18:50:49.768240 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.768221 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 18:50:49.768438 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.768422 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.768631 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.768614 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.768787 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.768767 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:49.768866 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.768847 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:50:49.769924 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.769896 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.773301 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.772448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.773301 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.772958 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.773301 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.773289 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.773818 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.773392 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xm86p\"" Jul 23 18:50:49.774079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.774061 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.776158 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.775984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.776158 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776096 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rk64l\"" Jul 23 18:50:49.776317 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-host\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.776317 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-serviceca\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.776430 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg62c\" (UniqueName: \"kubernetes.io/projected/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-kube-api-access-pg62c\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.776430 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d49df187-4039-4b20-b24f-9eb4e33d5620-agent-certs\") pod \"konnectivity-agent-4c2k5\" (UID: \"d49df187-4039-4b20-b24f-9eb4e33d5620\") " pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.776430 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d49df187-4039-4b20-b24f-9eb4e33d5620-konnectivity-ca\") pod \"konnectivity-agent-4c2k5\" (UID: \"d49df187-4039-4b20-b24f-9eb4e33d5620\") " pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.776771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776723 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.777013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.776979 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.778010 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.777976 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 18:50:49.778450 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.778197 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 18:50:49.778806 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.778650 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:49.778806 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.778718 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:50:49.780205 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.780185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.781977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.781251 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 18:50:49.781977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.781262 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 18:50:49.781977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.781498 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 18:50:49.781977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.781650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 18:50:49.781977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.781701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s4w2f\"" Jul 23 18:50:49.783846 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.783717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.788590 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.787855 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.788590 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.788032 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.788590 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.788260 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 18:50:49.788590 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.788419 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.788590 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.788447 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.788864 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.788819 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4vddh\"" Jul 23 18:50:49.788987 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.788966 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 18:50:49.789112 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.789004 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.789205 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.789175 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 18:50:49.789363 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.789327 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c5c7c\"" Jul 23 18:50:49.789831 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.789781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.790096 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.790079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 18:50:49.790571 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.790549 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gc7qg\"" Jul 23 18:50:49.790806 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.790567 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 18:50:49.790921 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.790583 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.792642 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.792623 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 18:50:49.792725 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.792701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qx729\"" Jul 23 18:50:49.795068 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.795026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 18:50:49.800605 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.800585 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dcjh8" Jul 23 18:50:49.810306 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.810279 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dcjh8" Jul 23 18:50:49.863956 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.863930 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 18:50:49.877313 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-var-lib-kubelet\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1e9591b-22a2-4007-ab06-ee4231ab97c5-iptables-alerter-script\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-registration-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7cms\" (UniqueName: \"kubernetes.io/projected/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-kube-api-access-k7cms\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-env-overrides\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrzvl\" (UniqueName: \"kubernetes.io/projected/2d958379-74e0-46ea-92c9-ff90d2b10d0a-kube-api-access-xrzvl\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.877443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-system-cni-dir\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cnibin\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877543 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-os-release\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-slash\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d49df187-4039-4b20-b24f-9eb4e33d5620-konnectivity-ca\") pod \"konnectivity-agent-4c2k5\" (UID: \"d49df187-4039-4b20-b24f-9eb4e33d5620\") " pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg62c\" (UniqueName: \"kubernetes.io/projected/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-kube-api-access-pg62c\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-kubernetes\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.877753 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-systemd\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/237ad0b3-52b5-4fc2-822d-4db074fb84aa-tmp-dir\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cpz6\" (UniqueName: \"kubernetes.io/projected/bfb1861c-a4c3-4224-87ba-df1e74c3df08-kube-api-access-4cpz6\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877900 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-run\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-lib-modules\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-ovn\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-log-socket\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.877990 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-cni-netd\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cpcq\" (UniqueName: \"kubernetes.io/projected/f7ffd29d-3d36-4699-b2a7-518022e8322e-kube-api-access-8cpcq\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-socket-dir-parent\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-hostroot\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-systemd-units\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a943505d-6390-4fe2-b647-70f281a4d58a-dbus\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-cnibin\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-daemon-config\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-multus-certs\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878204 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d49df187-4039-4b20-b24f-9eb4e33d5620-konnectivity-ca\") pod \"konnectivity-agent-4c2k5\" (UID: \"d49df187-4039-4b20-b24f-9eb4e33d5620\") " pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.878218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysctl-d\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kplb\" (UniqueName: \"kubernetes.io/projected/474bd293-5881-46ad-b608-d320abf9ce02-kube-api-access-8kplb\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-var-lib-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovnkube-config\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-socket-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-device-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysconfig\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-host\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878431 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-k8s-cni-cncf-io\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/474bd293-5881-46ad-b608-d320abf9ce02-etc-tuned\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-etc-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp9t8\" (UniqueName: \"kubernetes.io/projected/237ad0b3-52b5-4fc2-822d-4db074fb84aa-kube-api-access-bp9t8\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a943505d-6390-4fe2-b647-70f281a4d58a-kubelet-config\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878579 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cni-binary-copy\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.878697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-cni-bin\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1e9591b-22a2-4007-ab06-ee4231ab97c5-host-slash\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt4gf\" (UniqueName: \"kubernetes.io/projected/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-kube-api-access-bt4gf\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-node-log\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovn-node-metrics-cert\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/237ad0b3-52b5-4fc2-822d-4db074fb84aa-hosts-file\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-cni-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-conf-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-serviceca\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.878938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-sys\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/474bd293-5881-46ad-b608-d320abf9ce02-tmp\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwkc\" (UniqueName: \"kubernetes.io/projected/f1e9591b-22a2-4007-ab06-ee4231ab97c5-kube-api-access-fhwkc\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-cni-binary-copy\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-netns\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.879339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-cni-multus\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-kubelet\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-systemd\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-etc-kubernetes\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879307 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovnkube-script-lib\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879352 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-serviceca\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-etc-selinux\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-os-release\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-system-cni-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-run-netns\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-host\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-sys-fs\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-cni-bin\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-host\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d49df187-4039-4b20-b24f-9eb4e33d5620-agent-certs\") pod \"konnectivity-agent-4c2k5\" (UID: \"d49df187-4039-4b20-b24f-9eb4e33d5620\") " pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.880192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879654 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-modprobe-d\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.881049 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysctl-conf\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.881049 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-kubelet\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.881049 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.881049 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.879953 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 18:50:49.883948 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.883928 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d49df187-4039-4b20-b24f-9eb4e33d5620-agent-certs\") pod \"konnectivity-agent-4c2k5\" (UID: \"d49df187-4039-4b20-b24f-9eb4e33d5620\") " pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:49.892365 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.892342 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg62c\" (UniqueName: \"kubernetes.io/projected/a57ae30e-7a34-4d0e-84f3-90be4fe65f97-kube-api-access-pg62c\") pod \"node-ca-xlhsn\" (UID: \"a57ae30e-7a34-4d0e-84f3-90be4fe65f97\") " pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:49.922897 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.922813 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" event={"ID":"666bf80ba1e1983d2d3dd851bc8ab130","Type":"ContainerStarted","Data":"598a2ff15dc259a06e8d004a4d0bf16513013ad90479f176f270e07ce46d2ef4"} Jul 23 18:50:49.924036 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.924007 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" event={"ID":"3d37fd728dd468b7ee1c820042e75b2d","Type":"ContainerStarted","Data":"9d5f1eb079733ff4a084810a97fc89bb736f895829735cd198c8207796b915f8"} Jul 23 18:50:49.981103 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981053 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwkc\" (UniqueName: \"kubernetes.io/projected/f1e9591b-22a2-4007-ab06-ee4231ab97c5-kube-api-access-fhwkc\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-cni-binary-copy\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981151 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-netns\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-cni-multus\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-kubelet\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-systemd\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-etc-kubernetes\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:49.981321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovnkube-script-lib\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-etc-selinux\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981364 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-kubelet\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-os-release\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-system-cni-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-run-netns\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-sys-fs\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-cni-bin\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-modprobe-d\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysctl-conf\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-kubelet\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-var-lib-kubelet\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1e9591b-22a2-4007-ab06-ee4231ab97c5-iptables-alerter-script\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-registration-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7cms\" (UniqueName: \"kubernetes.io/projected/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-kube-api-access-k7cms\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.981809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-env-overrides\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrzvl\" (UniqueName: \"kubernetes.io/projected/2d958379-74e0-46ea-92c9-ff90d2b10d0a-kube-api-access-xrzvl\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-system-cni-dir\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cnibin\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.981990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-os-release\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-slash\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-kubernetes\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982101 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-cni-binary-copy\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-systemd\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/237ad0b3-52b5-4fc2-822d-4db074fb84aa-tmp-dir\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982180 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-netns\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cpz6\" (UniqueName: \"kubernetes.io/projected/bfb1861c-a4c3-4224-87ba-df1e74c3df08-kube-api-access-4cpz6\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-run\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982223 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-cni-multus\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-lib-modules\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.982673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-ovn\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982305 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-var-lib-kubelet\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982365 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-systemd\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982380 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-log-socket\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-cni-netd\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cpcq\" (UniqueName: \"kubernetes.io/projected/f7ffd29d-3d36-4699-b2a7-518022e8322e-kube-api-access-8cpcq\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.982479 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-socket-dir-parent\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982540 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-hostroot\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-systemd-units\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a943505d-6390-4fe2-b647-70f281a4d58a-dbus\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-etc-kubernetes\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-cnibin\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-daemon-config\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-multus-certs\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysctl-d\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kplb\" (UniqueName: \"kubernetes.io/projected/474bd293-5881-46ad-b608-d320abf9ce02-kube-api-access-8kplb\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-var-lib-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.983553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovnkube-config\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-socket-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982949 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-device-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysconfig\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-host\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/237ad0b3-52b5-4fc2-822d-4db074fb84aa-tmp-dir\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-k8s-cni-cncf-io\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/474bd293-5881-46ad-b608-d320abf9ce02-etc-tuned\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-etc-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-registration-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp9t8\" (UniqueName: \"kubernetes.io/projected/237ad0b3-52b5-4fc2-822d-4db074fb84aa-kube-api-access-bp9t8\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a943505d-6390-4fe2-b647-70f281a4d58a-kubelet-config\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cni-binary-copy\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983359 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1e9591b-22a2-4007-ab06-ee4231ab97c5-iptables-alerter-script\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.984553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-cni-bin\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1e9591b-22a2-4007-ab06-ee4231ab97c5-host-slash\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt4gf\" (UniqueName: \"kubernetes.io/projected/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-kube-api-access-bt4gf\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-node-log\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovn-node-metrics-cert\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/237ad0b3-52b5-4fc2-822d-4db074fb84aa-hosts-file\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-cni-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-conf-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983727 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-run\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-sys\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/474bd293-5881-46ad-b608-d320abf9ce02-tmp\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.983965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-lib-modules\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984005 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-env-overrides\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984020 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-ovn\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-log-socket\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.985409 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984148 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-system-cni-dir\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984193 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cnibin\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984329 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-cni-netd\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-socket-dir-parent\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984638 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-hostroot\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984683 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-systemd-units\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984818 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a943505d-6390-4fe2-b647-70f281a4d58a-dbus\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.984885 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:50.484853333 +0000 UTC m=+3.065997539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.984930 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.985055 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-os-release\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.985107 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-slash\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.985161 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-kubernetes\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.985217 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-systemd\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.985658 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986170 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986288 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-etc-selinux\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.986356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986355 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-os-release\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986402 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovnkube-script-lib\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986410 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-system-cni-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986467 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-run-netns\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986470 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-var-lib-cni-bin\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986556 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-modprobe-d\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysctl-conf\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986730 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-kubelet\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-var-lib-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986808 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-multus-certs\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.982706 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-cnibin\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.986951 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysctl-d\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987101 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-daemon-config\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987113 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/474bd293-5881-46ad-b608-d320abf9ce02-tmp\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-etc-sysconfig\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987180 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-device-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987241 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-host\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987282 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-host-run-k8s-cni-cncf-io\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.993529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-socket-dir\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f7ffd29d-3d36-4699-b2a7-518022e8322e-sys-fs\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1e9591b-22a2-4007-ab06-ee4231ab97c5-host-slash\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987473 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-etc-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.987747 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a943505d-6390-4fe2-b647-70f281a4d58a-kubelet-config\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.987833 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:49.987888 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:50:50.487870529 +0000 UTC m=+3.069014735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988184 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovnkube-config\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988378 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988384 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-cni-binary-copy\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988458 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-host-cni-bin\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988521 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/237ad0b3-52b5-4fc2-822d-4db074fb84aa-hosts-file\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988780 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-run-openvswitch\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.988840 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2d958379-74e0-46ea-92c9-ff90d2b10d0a-node-log\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.989243 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-conf-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.989310 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-multus-cni-dir\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.994498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.989443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/474bd293-5881-46ad-b608-d320abf9ce02-sys\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.995428 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.989781 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/474bd293-5881-46ad-b608-d320abf9ce02-etc-tuned\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:49.995428 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.991389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwkc\" (UniqueName: \"kubernetes.io/projected/f1e9591b-22a2-4007-ab06-ee4231ab97c5-kube-api-access-fhwkc\") pod \"iptables-alerter-2fwdn\" (UID: \"f1e9591b-22a2-4007-ab06-ee4231ab97c5\") " pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:49.995428 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.991718 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d958379-74e0-46ea-92c9-ff90d2b10d0a-ovn-node-metrics-cert\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:49.995587 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.995563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7cms\" (UniqueName: \"kubernetes.io/projected/8cf5a5e7-5d1d-4847-a01d-41e3babe2687-kube-api-access-k7cms\") pod \"multus-8gshw\" (UID: \"8cf5a5e7-5d1d-4847-a01d-41e3babe2687\") " pod="openshift-multus/multus-8gshw" Jul 23 18:50:49.998531 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.998504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cpcq\" (UniqueName: \"kubernetes.io/projected/f7ffd29d-3d36-4699-b2a7-518022e8322e-kube-api-access-8cpcq\") pod \"aws-ebs-csi-driver-node-sbkmz\" (UID: \"f7ffd29d-3d36-4699-b2a7-518022e8322e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:49.998531 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.998504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cpz6\" (UniqueName: \"kubernetes.io/projected/bfb1861c-a4c3-4224-87ba-df1e74c3df08-kube-api-access-4cpz6\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:49.998699 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:49.998623 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrzvl\" (UniqueName: \"kubernetes.io/projected/2d958379-74e0-46ea-92c9-ff90d2b10d0a-kube-api-access-xrzvl\") pod \"ovnkube-node-6wwvr\" (UID: \"2d958379-74e0-46ea-92c9-ff90d2b10d0a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:50.006455 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.006142 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:50:50.006455 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.006180 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:50:50.006455 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.006196 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:50.006455 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.006258 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:50.506245021 +0000 UTC m=+3.087389240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:50.006968 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.006921 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kplb\" (UniqueName: \"kubernetes.io/projected/474bd293-5881-46ad-b608-d320abf9ce02-kube-api-access-8kplb\") pod \"tuned-gnlxr\" (UID: \"474bd293-5881-46ad-b608-d320abf9ce02\") " pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:50.007335 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.007301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp9t8\" (UniqueName: \"kubernetes.io/projected/237ad0b3-52b5-4fc2-822d-4db074fb84aa-kube-api-access-bp9t8\") pod \"node-resolver-8dcd6\" (UID: \"237ad0b3-52b5-4fc2-822d-4db074fb84aa\") " pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:50.007335 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.007326 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt4gf\" (UniqueName: \"kubernetes.io/projected/11ac8e55-c6dc-4da3-a212-5cc7d3c74b96-kube-api-access-bt4gf\") pod \"multus-additional-cni-plugins-7h7t6\" (UID: \"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96\") " pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:50.034961 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.034934 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:50:50.070715 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.070688 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:50:50.080660 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.080639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xlhsn" Jul 23 18:50:50.091594 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.091564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" Jul 23 18:50:50.100513 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.100490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" Jul 23 18:50:50.108348 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.108331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:50:50.117259 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.117238 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8gshw" Jul 23 18:50:50.126049 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.126027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2fwdn" Jul 23 18:50:50.134171 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.134000 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" Jul 23 18:50:50.141931 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.141909 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8dcd6" Jul 23 18:50:50.487646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.487557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:50.487820 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.487737 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:50.487904 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.487819 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:51.487793321 +0000 UTC m=+4.068937521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:50.588832 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.588789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:50.589004 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.588917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:50.589061 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.589037 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:50.589102 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.589084 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:50:50.589136 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.589107 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:50:50.589136 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.589118 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:50:51.589101958 +0000 UTC m=+4.170246163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:50.589136 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.589122 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:50.589247 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.589182 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:51.589162362 +0000 UTC m=+4.170306583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:50.812884 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.812758 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:45:50 +0000 UTC" deadline="2028-04-17 06:59:12.606254258 +0000 UTC" Jul 23 18:50:50.812884 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.812805 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15204h8m21.793454169s" Jul 23 18:50:50.827938 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.827899 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:50:50.916523 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.916494 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:50.916681 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.916608 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:50:50.916681 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:50.916658 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:50.916764 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:50.916746 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:50:50.944921 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:50.944877 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ac8e55_c6dc_4da3_a212_5cc7d3c74b96.slice/crio-3ccd1a790d543bfe141b015771f3b1cf7b053d78664f4507856921394bf1fd42 WatchSource:0}: Error finding container 3ccd1a790d543bfe141b015771f3b1cf7b053d78664f4507856921394bf1fd42: Status 404 returned error can't find the container with id 3ccd1a790d543bfe141b015771f3b1cf7b053d78664f4507856921394bf1fd42 Jul 23 18:50:50.952668 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:50.952535 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf5a5e7_5d1d_4847_a01d_41e3babe2687.slice/crio-c78a46dc7102e2dcdb2313d72a37f1cd852babb1cda42816b3ec6b32281162e6 WatchSource:0}: Error finding container c78a46dc7102e2dcdb2313d72a37f1cd852babb1cda42816b3ec6b32281162e6: Status 404 returned error can't find the container with id c78a46dc7102e2dcdb2313d72a37f1cd852babb1cda42816b3ec6b32281162e6 Jul 23 18:50:50.953464 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:50.953431 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ffd29d_3d36_4699_b2a7_518022e8322e.slice/crio-c2b8c7441baddbfbd2ad06b66264afab4c7308eaeb4f20e7b8a5419b3f79fc3d WatchSource:0}: Error finding container c2b8c7441baddbfbd2ad06b66264afab4c7308eaeb4f20e7b8a5419b3f79fc3d: Status 404 returned error can't find the container with id c2b8c7441baddbfbd2ad06b66264afab4c7308eaeb4f20e7b8a5419b3f79fc3d Jul 23 18:50:50.954218 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:50.954190 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57ae30e_7a34_4d0e_84f3_90be4fe65f97.slice/crio-a5999f14bb3decbeea1b147345734948aa067dbc4cf4a86164bca7c29281d919 WatchSource:0}: Error finding container a5999f14bb3decbeea1b147345734948aa067dbc4cf4a86164bca7c29281d919: Status 404 returned error can't find the container with id a5999f14bb3decbeea1b147345734948aa067dbc4cf4a86164bca7c29281d919 Jul 23 18:50:50.956195 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:50.956084 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237ad0b3_52b5_4fc2_822d_4db074fb84aa.slice/crio-e88b349336ef8b8b5391fffe6b289d3af308e84be528ca890b1e4e0a65d94618 WatchSource:0}: Error finding container e88b349336ef8b8b5391fffe6b289d3af308e84be528ca890b1e4e0a65d94618: Status 404 returned error can't find the container with id e88b349336ef8b8b5391fffe6b289d3af308e84be528ca890b1e4e0a65d94618 Jul 23 18:50:50.958425 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:50:50.956692 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e9591b_22a2_4007_ab06_ee4231ab97c5.slice/crio-0c74798ab5f65d62a7779d46d56f2514631636aed56275a979eaccd8d0052de3 WatchSource:0}: Error finding container 0c74798ab5f65d62a7779d46d56f2514631636aed56275a979eaccd8d0052de3: Status 404 returned error can't find the container with id 0c74798ab5f65d62a7779d46d56f2514631636aed56275a979eaccd8d0052de3 Jul 23 18:50:51.498271 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.497923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:51.498271 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.498256 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:51.498549 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.498379 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:53.498325461 +0000 UTC m=+6.079469676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:51.601524 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.601469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:51.601720 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.601563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:51.601720 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.601708 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:51.601813 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.601774 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:50:53.601754944 +0000 UTC m=+6.182899159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:51.602245 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.602227 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:50:51.602332 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.602253 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:50:51.602332 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.602267 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:51.602332 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.602317 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:53.602300292 +0000 UTC m=+6.183444568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:51.813517 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.813392 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:45:50 +0000 UTC" deadline="2028-01-11 10:20:29.998501393 +0000 UTC" Jul 23 18:50:51.813517 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.813463 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12879h29m38.185042819s" Jul 23 18:50:51.919746 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.919711 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:51.919948 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:51.919864 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:50:51.935891 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.935851 2578 generic.go:356] "Generic (PLEG): container finished" podID="666bf80ba1e1983d2d3dd851bc8ab130" containerID="6a64b10c47daabe7b1b3919b4f3f10d6d2d7c1eaf5161d86f01d2b413c4ec95d" exitCode=0 Jul 23 18:50:51.936056 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.935965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" event={"ID":"666bf80ba1e1983d2d3dd851bc8ab130","Type":"ContainerDied","Data":"6a64b10c47daabe7b1b3919b4f3f10d6d2d7c1eaf5161d86f01d2b413c4ec95d"} Jul 23 18:50:51.946752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.946703 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"1216f20119a3a8a0324f8d92f044cff5a105804c88e84c1e5104c5102c08d1f3"} Jul 23 18:50:51.957815 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.957762 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" event={"ID":"474bd293-5881-46ad-b608-d320abf9ce02","Type":"ContainerStarted","Data":"e15b22460ee6b997ef762b279f03e36cef85810a8769f8826a99a36c4b14cc69"} Jul 23 18:50:51.963129 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.963100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2fwdn" event={"ID":"f1e9591b-22a2-4007-ab06-ee4231ab97c5","Type":"ContainerStarted","Data":"0c74798ab5f65d62a7779d46d56f2514631636aed56275a979eaccd8d0052de3"} Jul 23 18:50:51.999843 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:51.999801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8dcd6" event={"ID":"237ad0b3-52b5-4fc2-822d-4db074fb84aa","Type":"ContainerStarted","Data":"e88b349336ef8b8b5391fffe6b289d3af308e84be528ca890b1e4e0a65d94618"} Jul 23 18:50:52.014563 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.014528 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xlhsn" event={"ID":"a57ae30e-7a34-4d0e-84f3-90be4fe65f97","Type":"ContainerStarted","Data":"a5999f14bb3decbeea1b147345734948aa067dbc4cf4a86164bca7c29281d919"} Jul 23 18:50:52.019468 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.019251 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4c2k5" event={"ID":"d49df187-4039-4b20-b24f-9eb4e33d5620","Type":"ContainerStarted","Data":"a20b43518139651a8a55e348dfc9614b1161bf5e8969d31f7cf1202090ce4f64"} Jul 23 18:50:52.042610 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.041823 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" event={"ID":"3d37fd728dd468b7ee1c820042e75b2d","Type":"ContainerStarted","Data":"f7a4282a01b4f07f78fac85d8754b94780836196e20e61c251907f42b18f6fe2"} Jul 23 18:50:52.052842 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.052792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" event={"ID":"f7ffd29d-3d36-4699-b2a7-518022e8322e","Type":"ContainerStarted","Data":"c2b8c7441baddbfbd2ad06b66264afab4c7308eaeb4f20e7b8a5419b3f79fc3d"} Jul 23 18:50:52.059795 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.059763 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8gshw" event={"ID":"8cf5a5e7-5d1d-4847-a01d-41e3babe2687","Type":"ContainerStarted","Data":"c78a46dc7102e2dcdb2313d72a37f1cd852babb1cda42816b3ec6b32281162e6"} Jul 23 18:50:52.075749 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.075674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerStarted","Data":"3ccd1a790d543bfe141b015771f3b1cf7b053d78664f4507856921394bf1fd42"} Jul 23 18:50:52.916718 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.916685 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:52.917217 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:52.916732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:52.917217 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:52.916832 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:50:52.917217 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:52.916955 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:50:53.091787 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:53.091543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" event={"ID":"666bf80ba1e1983d2d3dd851bc8ab130","Type":"ContainerStarted","Data":"04a331ee2e79e344e7d3252fbdea373568e345ea0e5fc35f96fa73c9421ff2f8"} Jul 23 18:50:53.110345 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:53.110284 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" podStartSLOduration=4.110261619 podStartE2EDuration="4.110261619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:50:52.059747901 +0000 UTC m=+4.640892125" watchObservedRunningTime="2026-07-23 18:50:53.110261619 +0000 UTC m=+5.691405843" Jul 23 18:50:53.522155 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:53.521507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:53.522155 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.521690 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:53.522155 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.521757 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:57.52173814 +0000 UTC m=+10.102882357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:53.622621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:53.622746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.622901 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.622967 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:50:57.622948012 +0000 UTC m=+10.204092236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.623498 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.623523 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.623536 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:53.623674 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.623589 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:50:57.623572014 +0000 UTC m=+10.204716217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:53.921341 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:53.921041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:53.921807 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:53.921481 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:50:54.918026 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:54.917173 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:54.918026 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:54.917320 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:50:54.918026 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:54.917847 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:54.918026 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:54.917975 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:50:55.917760 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:55.917227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:55.917760 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:55.917386 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:50:56.917522 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:56.917483 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:56.917690 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:56.917496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:56.917690 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:56.917634 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:50:56.917777 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:56.917704 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:50:57.556710 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:57.556665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:57.557229 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.556866 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:57.557229 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.556958 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:05.556939589 +0000 UTC m=+18.138083793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:50:57.657192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:57.657146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:57.657392 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:57.657230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:57.657461 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.657410 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:50:57.657461 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.657429 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:50:57.657461 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.657444 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:57.657613 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.657506 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:05.657485776 +0000 UTC m=+18.238629995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:50:57.657965 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.657945 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:57.658063 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.658010 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:51:05.657991183 +0000 UTC m=+18.239135385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:50:57.923380 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:57.923297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:57.923542 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:57.923444 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:50:58.916929 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:58.916799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:50:58.917377 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:58.916937 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:50:58.917377 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:58.917325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:50:58.917482 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:58.917413 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:50:59.920472 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:50:59.920405 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:50:59.920856 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:50:59.920582 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:00.917194 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:00.917128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:00.917194 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:00.917146 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:00.917417 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:00.917247 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:00.917580 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:00.917542 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:01.917100 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:01.917068 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:01.917500 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:01.917193 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:02.917089 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:02.917048 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:02.917230 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:02.917096 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:02.917230 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:02.917154 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:02.917560 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:02.917289 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:03.917351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:03.917315 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:03.917815 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:03.917453 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:04.917066 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:04.917034 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:04.917303 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:04.917036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:04.917303 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:04.917153 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:04.917303 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:04.917243 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:05.621163 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:05.621125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:05.621339 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.621270 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:51:05.621339 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.621332 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:21.621315511 +0000 UTC m=+34.202459724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:51:05.721680 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:05.721643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:05.721897 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.721786 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:51:05.721897 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:05.721856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:05.721897 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.721876 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:51:21.721856393 +0000 UTC m=+34.303000591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:51:05.722119 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.721975 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:51:05.722119 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.721998 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:51:05.722119 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.722011 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:51:05.722119 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.722068 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:21.722052631 +0000 UTC m=+34.303196839 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:51:05.917398 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:05.917309 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:05.917912 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:05.917449 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:06.917005 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:06.916972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:06.917171 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:06.916972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:06.917171 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:06.917095 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:06.917171 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:06.917152 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:07.918354 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:07.917852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:07.918354 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:07.917987 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:08.916928 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:08.916893 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:08.916928 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:08.916919 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:08.917171 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:08.917016 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:08.917171 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:08.917151 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:09.917645 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:09.917611 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:09.918322 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:09.917743 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:10.919576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:10.919015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:10.919576 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:10.919167 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:10.919576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:10.919203 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:10.919576 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:10.919300 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:11.132002 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.131809 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" event={"ID":"f7ffd29d-3d36-4699-b2a7-518022e8322e","Type":"ContainerStarted","Data":"8b1e225f29a01c63ae498495c12bf90b7d71f74b691149ccf438023962048ce3"} Jul 23 18:51:11.134294 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.134271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8gshw" event={"ID":"8cf5a5e7-5d1d-4847-a01d-41e3babe2687","Type":"ContainerStarted","Data":"dfaddf864f087de405e22bd9e2a5d0a112e23f834a2ee408d030649a89934bfd"} Jul 23 18:51:11.136075 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.136029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerStarted","Data":"7e0d00e64c85b19c15a5647246933b3327c8cdb8615e49fa51e260a471a8bada"} Jul 23 18:51:11.138562 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.138537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"f2b854cbd20f7ff8bc08b097e4ea4f50d067821977216d13727f002f3a0f2826"} Jul 23 18:51:11.138636 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.138619 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"88595419cc027f710c4be4512f7387f4def7fbc446051ff7bfca16c588808097"} Jul 23 18:51:11.140509 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.140415 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" event={"ID":"474bd293-5881-46ad-b608-d320abf9ce02","Type":"ContainerStarted","Data":"02b713d84b9fab059811d57d724fc7e4b0c799ce910698ee5c09709cda5b0b19"} Jul 23 18:51:11.142479 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.142455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8dcd6" event={"ID":"237ad0b3-52b5-4fc2-822d-4db074fb84aa","Type":"ContainerStarted","Data":"245bb45de2186d87f697e7c6cfbfb67b7d934456c71d33dcd4d3588c0848faf2"} Jul 23 18:51:11.144372 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.144345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xlhsn" event={"ID":"a57ae30e-7a34-4d0e-84f3-90be4fe65f97","Type":"ContainerStarted","Data":"3f201d04e0fbab1cd6e8b5ae32713420a58d2ec4f9b68d6bb8cc9eb82bc47946"} Jul 23 18:51:11.145673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.145649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4c2k5" event={"ID":"d49df187-4039-4b20-b24f-9eb4e33d5620","Type":"ContainerStarted","Data":"b9b0460ba40f152d546986f342f2af8213f49a03139120752f72c769372455ff"} Jul 23 18:51:11.161442 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.161273 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" podStartSLOduration=22.16126242 podStartE2EDuration="22.16126242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:50:53.111569763 +0000 UTC m=+5.692713988" watchObservedRunningTime="2026-07-23 18:51:11.16126242 +0000 UTC m=+23.742406639" Jul 23 18:51:11.207448 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.207409 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8gshw" podStartSLOduration=3.248346812 podStartE2EDuration="23.207394113s" totalImagesPullingTime="19.959047301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:11.184382858 +0000 UTC m=+23.765527079" watchObservedRunningTime="2026-07-23 18:51:11.207394113 +0000 UTC m=+23.788538330" Jul 23 18:51:11.207745 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.207723 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4c2k5" podStartSLOduration=3.360012877 podStartE2EDuration="23.207717448s" totalImagesPullingTime="19.847704571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:11.207066163 +0000 UTC m=+23.788210382" watchObservedRunningTime="2026-07-23 18:51:11.207717448 +0000 UTC m=+23.788861667" Jul 23 18:51:11.247068 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.247020 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8dcd6" podStartSLOduration=3.349766791 podStartE2EDuration="23.246999759s" totalImagesPullingTime="19.897232968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:11.229796315 +0000 UTC m=+23.810940528" watchObservedRunningTime="2026-07-23 18:51:11.246999759 +0000 UTC m=+23.828143979" Jul 23 18:51:11.247189 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.247107 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xlhsn" podStartSLOduration=3.354097273 podStartE2EDuration="23.247101102s" totalImagesPullingTime="19.893003829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:11.246954673 +0000 UTC m=+23.828098893" watchObservedRunningTime="2026-07-23 18:51:11.247101102 +0000 UTC m=+23.828245321" Jul 23 18:51:11.340669 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.340644 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:11.413858 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.413813 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:11.515091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.515054 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:11.592236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.592210 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:11.915904 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.915869 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:11.917240 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:11.916805 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:11.917240 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:11.916964 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:12.018651 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.018618 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:12.149434 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.149352 2578 generic.go:356] "Generic (PLEG): container finished" podID="11ac8e55-c6dc-4da3-a212-5cc7d3c74b96" containerID="7e0d00e64c85b19c15a5647246933b3327c8cdb8615e49fa51e260a471a8bada" exitCode=0 Jul 23 18:51:12.149598 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.149431 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerDied","Data":"7e0d00e64c85b19c15a5647246933b3327c8cdb8615e49fa51e260a471a8bada"} Jul 23 18:51:12.152933 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.152897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"97599675fbc01fa29fe1c0bf18096e2904d0dc1b79dc417220d4cdff7fb9c033"} Jul 23 18:51:12.153122 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.152987 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"19463eaef84362def7fe183241d2d6a804f7a611bcdbab227801470bb907232c"} Jul 23 18:51:12.153122 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.153038 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"762f7d00af4f71acb982f0e08d6926d88b71be702fdff203b4e1c786de147bc5"} Jul 23 18:51:12.153122 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.153083 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"0a4814f9ace527742839df9bf1af4c3cca085a71ad0049a4195329c4dee3c39f"} Jul 23 18:51:12.437027 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.437006 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 18:51:12.836364 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.836210 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T18:51:12.43702328Z","UUID":"bbed9526-a515-41d5-b878-601faa24d280","Handler":null,"Name":"","Endpoint":""} Jul 23 18:51:12.837835 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.837785 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 18:51:12.837961 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.837849 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 18:51:12.917688 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.917635 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:12.917885 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:12.917768 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:12.918137 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.918114 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:12.918241 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:12.918222 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:12.955471 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:12.955398 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:13.029147 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.029113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:13.157262 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.157184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2fwdn" event={"ID":"f1e9591b-22a2-4007-ab06-ee4231ab97c5","Type":"ContainerStarted","Data":"12592eea5cdcf90e84e0a49fa4b214365906a5468254a3f9fc59698079c320a5"} Jul 23 18:51:13.159554 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.159530 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" event={"ID":"f7ffd29d-3d36-4699-b2a7-518022e8322e","Type":"ContainerStarted","Data":"acef6ad594ea15f1dc9789e3dd6758d8c73c3122150a02656f082570efbe0dbd"} Jul 23 18:51:13.184713 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.184662 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2fwdn" podStartSLOduration=5.289333758 podStartE2EDuration="25.184644659s" totalImagesPullingTime="19.895310901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:13.184337491 +0000 UTC m=+25.765481711" watchObservedRunningTime="2026-07-23 18:51:13.184644659 +0000 UTC m=+25.765788883" Jul 23 18:51:13.184887 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.184850 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gnlxr" podStartSLOduration=5.272030214 podStartE2EDuration="25.184839267s" totalImagesPullingTime="19.912809053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:11.270855642 +0000 UTC m=+23.851999863" watchObservedRunningTime="2026-07-23 18:51:13.184839267 +0000 UTC m=+25.765983479" Jul 23 18:51:13.311446 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.311413 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:13.411130 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.411052 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:13.513442 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.513414 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:13.917046 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:13.917011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:13.917195 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:13.917156 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:14.165539 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.165502 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"57ff052257f9d3ae8220ea18579acb76e68b4c00984f9f8a807f9a63b21617ff"} Jul 23 18:51:14.167848 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.167769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" event={"ID":"f7ffd29d-3d36-4699-b2a7-518022e8322e","Type":"ContainerStarted","Data":"be7c0f54184b8601ee8f672d4b55d02add39f920ba74f1762d36745de8ebf76e"} Jul 23 18:51:14.190056 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.190016 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbkmz" podStartSLOduration=3.970576661 podStartE2EDuration="26.189995832s" totalImagesPullingTime="22.219419171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:14.189255615 +0000 UTC m=+26.770399862" watchObservedRunningTime="2026-07-23 18:51:14.189995832 +0000 UTC m=+26.771140055" Jul 23 18:51:14.522552 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.522463 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:51:14.523123 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.523101 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:51:14.916801 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.916719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:14.916980 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:14.916722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:14.916980 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:14.916847 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:14.916980 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:14.916948 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:15.170842 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:15.170749 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:51:15.171304 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:15.171285 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4c2k5" Jul 23 18:51:15.294616 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:15.294588 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:15.917497 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:15.917362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:15.917619 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:15.917602 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:16.176210 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.176131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" event={"ID":"2d958379-74e0-46ea-92c9-ff90d2b10d0a","Type":"ContainerStarted","Data":"06e36fffb8f4af41aecc6970331b0ba897ef2a6fed8cd5ca9a77e0dec2a0a5b3"} Jul 23 18:51:16.177178 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.176363 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:51:16.177178 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.176429 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:51:16.177178 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.176471 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:51:16.178385 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.178359 2578 generic.go:356] "Generic (PLEG): container finished" podID="11ac8e55-c6dc-4da3-a212-5cc7d3c74b96" containerID="fb50ce0e8081bb05c33eb1e159731772a1a79b1ec3e8db6eedae81600391a098" exitCode=0 Jul 23 18:51:16.178497 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.178440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerDied","Data":"fb50ce0e8081bb05c33eb1e159731772a1a79b1ec3e8db6eedae81600391a098"} Jul 23 18:51:16.193277 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.193253 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:51:16.193485 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.193377 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:51:16.255668 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.255619 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" podStartSLOduration=8.294687628 podStartE2EDuration="28.255604521s" totalImagesPullingTime="19.960916893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:16.219144319 +0000 UTC m=+28.800288538" watchObservedRunningTime="2026-07-23 18:51:16.255604521 +0000 UTC m=+28.836748741" Jul 23 18:51:16.609755 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.609679 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:16.693840 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.693784 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:16.917040 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.916948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:16.917040 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:16.916987 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:16.917267 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:16.917096 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:16.917318 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:16.917259 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:17.918064 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:17.917931 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:17.918478 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:17.918128 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:18.102767 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.102734 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wphxg"] Jul 23 18:51:18.102951 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.102870 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:18.103020 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:18.102975 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:18.106163 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.106128 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wkn9h"] Jul 23 18:51:18.106293 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.106220 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:18.106354 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:18.106314 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:18.111313 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.111266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zbwbh"] Jul 23 18:51:18.184780 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.184757 2578 generic.go:356] "Generic (PLEG): container finished" podID="11ac8e55-c6dc-4da3-a212-5cc7d3c74b96" containerID="abe0e3eb0265b0a4ef5a7a32457c7db4a75f133389da789f2e9a189cbe1d244a" exitCode=0 Jul 23 18:51:18.184921 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.184810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:18.184963 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.184913 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerDied","Data":"abe0e3eb0265b0a4ef5a7a32457c7db4a75f133389da789f2e9a189cbe1d244a"} Jul 23 18:51:18.184963 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:18.184934 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:18.377615 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.377536 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:18.470183 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.470147 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:18.570773 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:18.570746 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:19.406965 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:19.406940 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:19.511976 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:19.511945 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:19.603048 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:19.603021 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:19.917094 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:19.917054 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:19.917094 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:19.917086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:19.917319 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:19.917054 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:19.917319 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:19.917221 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:19.917319 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:19.917296 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:19.917461 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:19.917379 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:20.084935 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:20.084903 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:20.168196 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:20.168115 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:20.191202 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:20.191168 2578 generic.go:356] "Generic (PLEG): container finished" podID="11ac8e55-c6dc-4da3-a212-5cc7d3c74b96" containerID="40640fb29ffb5b265273faad7f746b58736afaa00ed708ea3571d2de7dee2ebd" exitCode=0 Jul 23 18:51:20.191361 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:20.191207 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerDied","Data":"40640fb29ffb5b265273faad7f746b58736afaa00ed708ea3571d2de7dee2ebd"} Jul 23 18:51:21.161673 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.161640 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:21.241760 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.241729 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:21.513682 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.513601 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:21.634971 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.634937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:21.635134 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.635073 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:51:21.635134 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.635130 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs podName:bfb1861c-a4c3-4224-87ba-df1e74c3df08 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:53.635114566 +0000 UTC m=+66.216258904 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs") pod "network-metrics-daemon-zbwbh" (UID: "bfb1861c-a4c3-4224-87ba-df1e74c3df08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:51:21.736154 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.736124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:21.736309 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.736180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:21.736309 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.736295 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:51:21.736309 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.736298 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:51:21.736446 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.736319 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:51:21.736446 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.736330 2578 projected.go:196] Error preparing data for projected volume kube-api-access-k88ps for pod openshift-network-diagnostics/network-check-target-wkn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:51:21.736446 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.736359 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret podName:a943505d-6390-4fe2-b647-70f281a4d58a nodeName:}" failed. No retries permitted until 2026-07-23 18:51:53.736343365 +0000 UTC m=+66.317487581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret") pod "global-pull-secret-syncer-wphxg" (UID: "a943505d-6390-4fe2-b647-70f281a4d58a") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:51:21.736446 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.736376 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps podName:63e75e39-7a09-4754-bf45-8d687b92ee09 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:53.736368853 +0000 UTC m=+66.317513051 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k88ps" (UniqueName: "kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps") pod "network-check-target-wkn9h" (UID: "63e75e39-7a09-4754-bf45-8d687b92ee09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:51:21.802573 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.802489 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:21.813510 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.813486 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8dcd6_237ad0b3-52b5-4fc2-822d-4db074fb84aa/dns-node-resolver/0.log" Jul 23 18:51:21.907274 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.907233 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:21.917977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.917951 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:21.917977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.917966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:21.918162 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.918053 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wphxg" podUID="a943505d-6390-4fe2-b647-70f281a4d58a" Jul 23 18:51:21.918162 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.918126 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbwbh" podUID="bfb1861c-a4c3-4224-87ba-df1e74c3df08" Jul 23 18:51:21.918162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:21.918138 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:21.918311 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:21.918213 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wkn9h" podUID="63e75e39-7a09-4754-bf45-8d687b92ee09" Jul 23 18:51:23.016718 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.016683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xlhsn_a57ae30e-7a34-4d0e-84f3-90be4fe65f97/node-ca/0.log" Jul 23 18:51:23.374548 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.374476 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:23.443866 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.443805 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:23.526932 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.526899 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:23.777879 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.777790 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeReady" Jul 23 18:51:23.778048 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.777945 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 18:51:23.799650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.799619 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:23.843856 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.843582 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d6htd"] Jul 23 18:51:23.849886 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.849411 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tghlx"] Jul 23 18:51:23.850319 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.850297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:23.851784 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.851763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:23.853155 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.853135 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6742f\"" Jul 23 18:51:23.853260 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.853197 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 18:51:23.853334 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.853320 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 18:51:23.853428 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.853414 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 18:51:23.854148 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.854003 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 18:51:23.854148 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.854120 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 18:51:23.854148 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.854120 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-66l9k\"" Jul 23 18:51:23.854398 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.854376 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 18:51:23.854854 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.854813 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 18:51:23.858436 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.858256 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d6htd"] Jul 23 18:51:23.862367 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.862346 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tghlx"] Jul 23 18:51:23.862545 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.862529 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bz472"] Jul 23 18:51:23.864568 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.864538 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bz472" Jul 23 18:51:23.868081 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.867912 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 18:51:23.868081 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.867933 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zxgvq\"" Jul 23 18:51:23.868208 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.868094 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 18:51:23.878191 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.878169 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bz472"] Jul 23 18:51:23.916932 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.916892 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:23.917157 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.917139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:23.917627 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.917599 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:23.921064 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.921043 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 18:51:23.921219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.921201 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mqhg9\"" Jul 23 18:51:23.921287 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.921268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8ssn\"" Jul 23 18:51:23.921344 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.921272 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 18:51:23.921631 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.921460 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 18:51:23.921631 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.921474 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 18:51:23.953109 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f0b072b-a731-4848-8feb-14c367e5f882-crio-socket\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:23.953219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:23.953219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953165 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wknjw\" (UniqueName: \"kubernetes.io/projected/9b992573-9754-42a5-af40-5aedcc85454b-kube-api-access-wknjw\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:23.953219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f0b072b-a731-4848-8feb-14c367e5f882-data-volume\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:23.953219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953217 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f387e34b-b873-48e9-8642-f31657c6309a-cert\") pod \"ingress-canary-d6htd\" (UID: \"f387e34b-b873-48e9-8642-f31657c6309a\") " pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:23.953400 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0b072b-a731-4848-8feb-14c367e5f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:23.953400 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgt6t\" (UniqueName: \"kubernetes.io/projected/f387e34b-b873-48e9-8642-f31657c6309a-kube-api-access-hgt6t\") pod \"ingress-canary-d6htd\" (UID: \"f387e34b-b873-48e9-8642-f31657c6309a\") " pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:23.953400 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953299 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b992573-9754-42a5-af40-5aedcc85454b-config-volume\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:23.953400 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b992573-9754-42a5-af40-5aedcc85454b-tmp-dir\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:23.953556 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:23.953556 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:23.953460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hkfc\" (UniqueName: \"kubernetes.io/projected/8f0b072b-a731-4848-8feb-14c367e5f882-kube-api-access-6hkfc\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.054002 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.053913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgt6t\" (UniqueName: \"kubernetes.io/projected/f387e34b-b873-48e9-8642-f31657c6309a-kube-api-access-hgt6t\") pod \"ingress-canary-d6htd\" (UID: \"f387e34b-b873-48e9-8642-f31657c6309a\") " pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:24.054002 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.053966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b992573-9754-42a5-af40-5aedcc85454b-config-volume\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b992573-9754-42a5-af40-5aedcc85454b-tmp-dir\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hkfc\" (UniqueName: \"kubernetes.io/projected/8f0b072b-a731-4848-8feb-14c367e5f882-kube-api-access-6hkfc\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f0b072b-a731-4848-8feb-14c367e5f882-crio-socket\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wknjw\" (UniqueName: \"kubernetes.io/projected/9b992573-9754-42a5-af40-5aedcc85454b-kube-api-access-wknjw\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.054396 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f0b072b-a731-4848-8feb-14c367e5f882-data-volume\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.054451 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls podName:9b992573-9754-42a5-af40-5aedcc85454b nodeName:}" failed. No retries permitted until 2026-07-23 18:51:24.554430819 +0000 UTC m=+37.135575017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls") pod "dns-default-bz472" (UID: "9b992573-9754-42a5-af40-5aedcc85454b") : secret "dns-default-metrics-tls" not found Jul 23 18:51:24.054480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f387e34b-b873-48e9-8642-f31657c6309a-cert\") pod \"ingress-canary-d6htd\" (UID: \"f387e34b-b873-48e9-8642-f31657c6309a\") " pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:24.054991 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0b072b-a731-4848-8feb-14c367e5f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.054991 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.054745 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f0b072b-a731-4848-8feb-14c367e5f882-data-volume\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.055091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.055055 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b992573-9754-42a5-af40-5aedcc85454b-config-volume\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.055134 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.055095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f0b072b-a731-4848-8feb-14c367e5f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.055559 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.055532 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b992573-9754-42a5-af40-5aedcc85454b-tmp-dir\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.055685 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.055608 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f0b072b-a731-4848-8feb-14c367e5f882-crio-socket\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.055685 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.055633 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:51:24.055796 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.055694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls podName:8f0b072b-a731-4848-8feb-14c367e5f882 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:24.555677009 +0000 UTC m=+37.136821232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tghlx" (UID: "8f0b072b-a731-4848-8feb-14c367e5f882") : secret "insights-runtime-extractor-tls" not found Jul 23 18:51:24.060395 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.060374 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f387e34b-b873-48e9-8642-f31657c6309a-cert\") pod \"ingress-canary-d6htd\" (UID: \"f387e34b-b873-48e9-8642-f31657c6309a\") " pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:24.074762 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.074740 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hkfc\" (UniqueName: \"kubernetes.io/projected/8f0b072b-a731-4848-8feb-14c367e5f882-kube-api-access-6hkfc\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.075732 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.075714 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wknjw\" (UniqueName: \"kubernetes.io/projected/9b992573-9754-42a5-af40-5aedcc85454b-kube-api-access-wknjw\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.079424 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.077604 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:24.083236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.083210 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgt6t\" (UniqueName: \"kubernetes.io/projected/f387e34b-b873-48e9-8642-f31657c6309a-kube-api-access-hgt6t\") pod \"ingress-canary-d6htd\" (UID: \"f387e34b-b873-48e9-8642-f31657c6309a\") " pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:24.166908 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.166874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d6htd" Jul 23 18:51:24.177869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.177843 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:24.388314 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.388287 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d6htd"] Jul 23 18:51:24.393812 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:24.393766 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf387e34b_b873_48e9_8642_f31657c6309a.slice/crio-ebe0baa123bd66e4c632edae77321836fa9ff9da0e46d8f3c7c26a9dddc7751e WatchSource:0}: Error finding container ebe0baa123bd66e4c632edae77321836fa9ff9da0e46d8f3c7c26a9dddc7751e: Status 404 returned error can't find the container with id ebe0baa123bd66e4c632edae77321836fa9ff9da0e46d8f3c7c26a9dddc7751e Jul 23 18:51:24.476244 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.476218 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:24.557977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.557946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:24.558077 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:24.557995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:24.558131 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.558115 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:51:24.558184 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.558173 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls podName:9b992573-9754-42a5-af40-5aedcc85454b nodeName:}" failed. No retries permitted until 2026-07-23 18:51:25.558159581 +0000 UTC m=+38.139303779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls") pod "dns-default-bz472" (UID: "9b992573-9754-42a5-af40-5aedcc85454b") : secret "dns-default-metrics-tls" not found Jul 23 18:51:24.558225 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.558115 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:51:24.558260 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:24.558227 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls podName:8f0b072b-a731-4848-8feb-14c367e5f882 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:25.558218367 +0000 UTC m=+38.139362570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tghlx" (UID: "8f0b072b-a731-4848-8feb-14c367e5f882") : secret "insights-runtime-extractor-tls" not found Jul 23 18:51:25.203750 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.203668 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d6htd" event={"ID":"f387e34b-b873-48e9-8642-f31657c6309a","Type":"ContainerStarted","Data":"ebe0baa123bd66e4c632edae77321836fa9ff9da0e46d8f3c7c26a9dddc7751e"} Jul 23 18:51:25.206798 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.206768 2578 generic.go:356] "Generic (PLEG): container finished" podID="11ac8e55-c6dc-4da3-a212-5cc7d3c74b96" containerID="f5ebd05cf8bacf67fcc937773c7cda6378838036ac92f955aac8a99ad80b3d74" exitCode=0 Jul 23 18:51:25.206945 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.206861 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerDied","Data":"f5ebd05cf8bacf67fcc937773c7cda6378838036ac92f955aac8a99ad80b3d74"} Jul 23 18:51:25.552917 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.552640 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:25.568778 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.568744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:25.568928 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.568870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:25.568974 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:25.568934 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:51:25.568974 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:25.568963 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:51:25.569078 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:25.569007 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls podName:8f0b072b-a731-4848-8feb-14c367e5f882 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:27.568987029 +0000 UTC m=+40.150131233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tghlx" (UID: "8f0b072b-a731-4848-8feb-14c367e5f882") : secret "insights-runtime-extractor-tls" not found Jul 23 18:51:25.569078 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:25.569030 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls podName:9b992573-9754-42a5-af40-5aedcc85454b nodeName:}" failed. No retries permitted until 2026-07-23 18:51:27.569014212 +0000 UTC m=+40.150158413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls") pod "dns-default-bz472" (UID: "9b992573-9754-42a5-af40-5aedcc85454b") : secret "dns-default-metrics-tls" not found Jul 23 18:51:25.731907 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.731877 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:25.837137 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.837074 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:25.936195 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:25.936146 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:26.052003 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:26.051968 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:26.213145 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:26.213027 2578 generic.go:356] "Generic (PLEG): container finished" podID="11ac8e55-c6dc-4da3-a212-5cc7d3c74b96" containerID="073a2ab202b1593df6e8472bffba65c9b181f3ec7420437bf323adf60b6c9b53" exitCode=0 Jul 23 18:51:26.213145 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:26.213097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerDied","Data":"073a2ab202b1593df6e8472bffba65c9b181f3ec7420437bf323adf60b6c9b53"} Jul 23 18:51:26.289647 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:26.289613 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:27.019193 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.019168 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:27.217263 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.217230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d6htd" event={"ID":"f387e34b-b873-48e9-8642-f31657c6309a","Type":"ContainerStarted","Data":"b9f136409fe2b12630d6672044f438bf58ea4a677f12bbf6add96b2a44c9824d"} Jul 23 18:51:27.220648 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.220621 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" event={"ID":"11ac8e55-c6dc-4da3-a212-5cc7d3c74b96","Type":"ContainerStarted","Data":"0c98f1b166a84ee015c7a9fdefd7183980f658a27cc666666659465381dfa363"} Jul 23 18:51:27.237051 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.237005 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d6htd" podStartSLOduration=1.699074889 podStartE2EDuration="4.236990375s" totalImagesPullingTime="2.537915486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:27.236445936 +0000 UTC m=+39.817590156" watchObservedRunningTime="2026-07-23 18:51:27.236990375 +0000 UTC m=+39.818134630" Jul 23 18:51:27.269298 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.269210 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7h7t6" podStartSLOduration=8.4161231 podStartE2EDuration="39.26919136s" totalImagesPullingTime="30.85306826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:27.266789851 +0000 UTC m=+39.847934076" watchObservedRunningTime="2026-07-23 18:51:27.26919136 +0000 UTC m=+39.850335580" Jul 23 18:51:27.586280 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.586246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:27.586433 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.586305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:27.586433 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:27.586391 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:51:27.586517 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:27.586458 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls podName:9b992573-9754-42a5-af40-5aedcc85454b nodeName:}" failed. No retries permitted until 2026-07-23 18:51:31.586433499 +0000 UTC m=+44.167577697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls") pod "dns-default-bz472" (UID: "9b992573-9754-42a5-af40-5aedcc85454b") : secret "dns-default-metrics-tls" not found Jul 23 18:51:27.586517 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:27.586391 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:51:27.586593 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:27.586554 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls podName:8f0b072b-a731-4848-8feb-14c367e5f882 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:31.586519162 +0000 UTC m=+44.167663363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tghlx" (UID: "8f0b072b-a731-4848-8feb-14c367e5f882") : secret "insights-runtime-extractor-tls" not found Jul 23 18:51:27.704895 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.704862 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:27.843346 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.843269 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:27.941090 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:27.941027 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:28.040133 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:28.040100 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:28.115441 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:28.115366 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:28.241443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:28.241414 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:28.841287 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:28.841254 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:29.195573 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:29.195500 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:29.289485 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:29.289452 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:29.424057 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:29.424024 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:29.525251 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:29.525181 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:29.644395 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:29.644364 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:29.831963 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:29.831932 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:30.054354 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:30.054324 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:30.185922 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:30.185839 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:30.288526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:30.288491 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:31.022464 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.022435 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:31.614630 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.614593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:31.614801 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.614674 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:31.618384 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.618358 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b992573-9754-42a5-af40-5aedcc85454b-metrics-tls\") pod \"dns-default-bz472\" (UID: \"9b992573-9754-42a5-af40-5aedcc85454b\") " pod="openshift-dns/dns-default-bz472" Jul 23 18:51:31.618499 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.618360 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f0b072b-a731-4848-8feb-14c367e5f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tghlx\" (UID: \"8f0b072b-a731-4848-8feb-14c367e5f882\") " pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:31.637785 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.637759 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:31.672782 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.672765 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tghlx" Jul 23 18:51:31.679917 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.679897 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bz472" Jul 23 18:51:31.853946 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.853919 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:31.886961 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.886890 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tghlx"] Jul 23 18:51:31.890641 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:31.890616 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0b072b_a731_4848_8feb_14c367e5f882.slice/crio-e2f449738d4226426018481728bc66ff768dbc055bf1faf310351bb8596125fe WatchSource:0}: Error finding container e2f449738d4226426018481728bc66ff768dbc055bf1faf310351bb8596125fe: Status 404 returned error can't find the container with id e2f449738d4226426018481728bc66ff768dbc055bf1faf310351bb8596125fe Jul 23 18:51:31.913689 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:31.913667 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bz472"] Jul 23 18:51:31.916136 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:31.916117 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b992573_9754_42a5_af40_5aedcc85454b.slice/crio-aa92e44a689f6aa250211c15a3d5f5ae39a91e9789f5603ad65edce391359acb WatchSource:0}: Error finding container aa92e44a689f6aa250211c15a3d5f5ae39a91e9789f5603ad65edce391359acb: Status 404 returned error can't find the container with id aa92e44a689f6aa250211c15a3d5f5ae39a91e9789f5603ad65edce391359acb Jul 23 18:51:32.067932 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:32.067899 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:32.229642 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:32.229560 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:32.233283 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:32.233259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bz472" event={"ID":"9b992573-9754-42a5-af40-5aedcc85454b","Type":"ContainerStarted","Data":"aa92e44a689f6aa250211c15a3d5f5ae39a91e9789f5603ad65edce391359acb"} Jul 23 18:51:32.234332 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:32.234311 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tghlx" event={"ID":"8f0b072b-a731-4848-8feb-14c367e5f882","Type":"ContainerStarted","Data":"e2f449738d4226426018481728bc66ff768dbc055bf1faf310351bb8596125fe"} Jul 23 18:51:32.333834 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:32.333791 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:33.234213 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:33.234177 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:33.631091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:33.631053 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:33.828014 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:33.827983 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:34.035295 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:34.035218 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:34.322276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:34.322255 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:34.811726 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:34.811692 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:34.919718 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:34.919686 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:35.244092 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.244060 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:35.244521 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.244500 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bz472" event={"ID":"9b992573-9754-42a5-af40-5aedcc85454b","Type":"ContainerStarted","Data":"f78bca904b2a6d82a29c987ec2ec6fa064dea01ed689a18cf9094888b0f501d1"} Jul 23 18:51:35.244638 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.244552 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bz472" event={"ID":"9b992573-9754-42a5-af40-5aedcc85454b","Type":"ContainerStarted","Data":"72dc8962f54da236ff2fe3ed5de6a8ef0fdc3e49e93c3127a9689644a108c56e"} Jul 23 18:51:35.244638 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.244626 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bz472" Jul 23 18:51:35.631703 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.631665 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:35.828247 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.828196 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:35.960859 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:35.960762 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:36.035340 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:36.035307 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:36.225121 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:36.225045 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:39.255606 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:39.255570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tghlx" event={"ID":"8f0b072b-a731-4848-8feb-14c367e5f882","Type":"ContainerStarted","Data":"5a4e1e5718692d9e5623b50553b362a5ce7130b0924b1fa7cd7feb28aa588a6c"} Jul 23 18:51:43.019895 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:43.019859 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:43.226813 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:43.226711 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:43.269121 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:43.269072 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tghlx" event={"ID":"8f0b072b-a731-4848-8feb-14c367e5f882","Type":"ContainerStarted","Data":"f9c55df6e3666b1870cf607b24c8af97d49ccce38f608124551e1402aa2dd949"} Jul 23 18:51:45.249389 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:45.249347 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bz472" Jul 23 18:51:45.279246 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:45.278684 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bz472" podStartSLOduration=20.024677656 podStartE2EDuration="22.278656227s" totalImagesPullingTime="2.253978571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:35.268585786 +0000 UTC m=+47.849730006" watchObservedRunningTime="2026-07-23 18:51:45.278656227 +0000 UTC m=+57.859800448" Jul 23 18:51:46.278235 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:46.278207 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tghlx" event={"ID":"8f0b072b-a731-4848-8feb-14c367e5f882","Type":"ContainerStarted","Data":"03b191de512e57ab269d13f8a63caf8b759f93c48e706048fab56c380e2ed116"} Jul 23 18:51:46.307416 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:46.307369 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tghlx" podStartSLOduration=9.169476808 podStartE2EDuration="23.307351484s" totalImagesPullingTime="14.137874676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:51:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:46.305717814 +0000 UTC m=+58.886862035" watchObservedRunningTime="2026-07-23 18:51:46.307351484 +0000 UTC m=+58.888495704" Jul 23 18:51:48.196229 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:48.196194 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wwvr" Jul 23 18:51:50.464501 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.464471 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r"] Jul 23 18:51:50.467013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.466994 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.471209 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.471186 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d7rrv\"" Jul 23 18:51:50.471301 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.471236 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 18:51:50.471963 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.471947 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 18:51:50.541349 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.541326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r"] Jul 23 18:51:50.550745 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.550717 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67b5b58d9c-6r4qf"] Jul 23 18:51:50.552353 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.552338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.555265 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.555242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5dfbf97b-129b-48f7-a0df-09148ecf8e6b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rmj4r\" (UID: \"5dfbf97b-129b-48f7-a0df-09148ecf8e6b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.555358 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.555268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5dfbf97b-129b-48f7-a0df-09148ecf8e6b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rmj4r\" (UID: \"5dfbf97b-129b-48f7-a0df-09148ecf8e6b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.557222 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.557200 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 18:51:50.557309 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.557243 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 18:51:50.557376 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.557333 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gbdxn\"" Jul 23 18:51:50.558587 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.558559 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 18:51:50.566205 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.566188 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 18:51:50.584119 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.584093 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b5b58d9c-6r4qf"] Jul 23 18:51:50.655571 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-image-registry-private-configuration\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655667 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-trusted-ca\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655667 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-tls\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655667 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c83006d-4a76-4def-96fe-755b5a6da28b-ca-trust-extracted\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655762 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-installation-pull-secrets\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655762 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxgls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-kube-api-access-sxgls\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655762 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5dfbf97b-129b-48f7-a0df-09148ecf8e6b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rmj4r\" (UID: \"5dfbf97b-129b-48f7-a0df-09148ecf8e6b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.655762 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5dfbf97b-129b-48f7-a0df-09148ecf8e6b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rmj4r\" (UID: \"5dfbf97b-129b-48f7-a0df-09148ecf8e6b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.655941 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-certificates\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.655941 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.655807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-bound-sa-token\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.656397 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.656377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5dfbf97b-129b-48f7-a0df-09148ecf8e6b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rmj4r\" (UID: \"5dfbf97b-129b-48f7-a0df-09148ecf8e6b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.658215 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.658198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5dfbf97b-129b-48f7-a0df-09148ecf8e6b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rmj4r\" (UID: \"5dfbf97b-129b-48f7-a0df-09148ecf8e6b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.684342 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.684319 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b5b58d9c-6r4qf"] Jul 23 18:51:50.684500 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:50.684482 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-sxgls registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" podUID="8c83006d-4a76-4def-96fe-755b5a6da28b" Jul 23 18:51:50.699392 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.699345 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:50.756280 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-certificates\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756280 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-bound-sa-token\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756419 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-image-registry-private-configuration\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756419 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-trusted-ca\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756539 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-tls\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756594 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c83006d-4a76-4def-96fe-755b5a6da28b-ca-trust-extracted\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756594 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756576 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-installation-pull-secrets\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.756689 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxgls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-kube-api-access-sxgls\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.757001 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.756972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c83006d-4a76-4def-96fe-755b5a6da28b-ca-trust-extracted\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.757124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.757085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-certificates\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.757331 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.757313 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-trusted-ca\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.759198 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.759181 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-installation-pull-secrets\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.759450 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.759431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-tls\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.759490 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.759455 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-image-registry-private-configuration\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.772942 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.772915 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-bound-sa-token\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.775635 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.775620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" Jul 23 18:51:50.776135 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.776114 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxgls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-kube-api-access-sxgls\") pod \"image-registry-67b5b58d9c-6r4qf\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:50.870783 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.870750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:50.929732 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:50.929704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r"] Jul 23 18:51:50.933200 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:50.933176 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfbf97b_129b_48f7_a0df_09148ecf8e6b.slice/crio-d8c35e449f77f16e15e319e97f7e03c2e7ecdac5b4190a38b92c6b17a2331102 WatchSource:0}: Error finding container d8c35e449f77f16e15e319e97f7e03c2e7ecdac5b4190a38b92c6b17a2331102: Status 404 returned error can't find the container with id d8c35e449f77f16e15e319e97f7e03c2e7ecdac5b4190a38b92c6b17a2331102 Jul 23 18:51:51.085912 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.085881 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:51.293999 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.293968 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:51.293999 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.293976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" event={"ID":"5dfbf97b-129b-48f7-a0df-09148ecf8e6b","Type":"ContainerStarted","Data":"d8c35e449f77f16e15e319e97f7e03c2e7ecdac5b4190a38b92c6b17a2331102"} Jul 23 18:51:51.299696 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.299675 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:51.360960 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.360881 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-bound-sa-token\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.360960 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.360947 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-installation-pull-secrets\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.361140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.360976 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxgls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-kube-api-access-sxgls\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.361140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.361005 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-certificates\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.361140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.361035 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-image-registry-private-configuration\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.361140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.361067 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-trusted-ca\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.361140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.361092 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c83006d-4a76-4def-96fe-755b5a6da28b-ca-trust-extracted\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.361336 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.361143 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-tls\") pod \"8c83006d-4a76-4def-96fe-755b5a6da28b\" (UID: \"8c83006d-4a76-4def-96fe-755b5a6da28b\") " Jul 23 18:51:51.362185 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.362060 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:51:51.362315 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.362277 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:51:51.362384 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.362342 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c83006d-4a76-4def-96fe-755b5a6da28b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:51:51.363872 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.363844 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:51:51.363872 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.363859 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:51:51.364076 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.364012 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:51:51.364076 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.364031 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-kube-api-access-sxgls" (OuterVolumeSpecName: "kube-api-access-sxgls") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "kube-api-access-sxgls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:51:51.364522 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.364499 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8c83006d-4a76-4def-96fe-755b5a6da28b" (UID: "8c83006d-4a76-4def-96fe-755b5a6da28b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:51:51.462726 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462698 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-tls\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.462726 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462726 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-bound-sa-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.462726 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462734 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-installation-pull-secrets\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.463013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462743 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxgls\" (UniqueName: \"kubernetes.io/projected/8c83006d-4a76-4def-96fe-755b5a6da28b-kube-api-access-sxgls\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.463013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462753 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-registry-certificates\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.463013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462761 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c83006d-4a76-4def-96fe-755b5a6da28b-image-registry-private-configuration\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.463013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462769 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c83006d-4a76-4def-96fe-755b5a6da28b-trusted-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.463013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.462777 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c83006d-4a76-4def-96fe-755b5a6da28b-ca-trust-extracted\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:51:51.937745 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:51.937719 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:52.097089 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.097041 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:52.298157 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.298070 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b5b58d9c-6r4qf" Jul 23 18:51:52.298157 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.298070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" event={"ID":"5dfbf97b-129b-48f7-a0df-09148ecf8e6b","Type":"ContainerStarted","Data":"5f53509d9508e695550dfebfe49483f085863db3829c7df003c66dee0df94cc5"} Jul 23 18:51:52.340422 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.340377 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rmj4r" podStartSLOduration=1.388311325 podStartE2EDuration="2.3403622s" totalImagesPullingTime="952.050875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:52.340360443 +0000 UTC m=+64.921504662" watchObservedRunningTime="2026-07-23 18:51:52.3403622 +0000 UTC m=+64.921506419" Jul 23 18:51:52.346521 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.346498 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:52.407942 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.407906 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b5b58d9c-6r4qf"] Jul 23 18:51:52.424909 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.424877 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67b5b58d9c-6r4qf"] Jul 23 18:51:52.519103 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.519078 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:52.642657 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.642620 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:52.680199 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.680173 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst"] Jul 23 18:51:52.682041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.682027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:52.685750 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.685731 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mv9h5\"" Jul 23 18:51:52.685923 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.685904 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 23 18:51:52.698420 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.698400 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst"] Jul 23 18:51:52.755888 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.755860 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:52.771987 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.771966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/889397c4-85f2-41d2-8739-0f3591c12e03-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-26rst\" (UID: \"889397c4-85f2-41d2-8739-0f3591c12e03\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:52.872961 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:52.872930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/889397c4-85f2-41d2-8739-0f3591c12e03-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-26rst\" (UID: \"889397c4-85f2-41d2-8739-0f3591c12e03\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:52.873097 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:52.873073 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 23 18:51:52.873143 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:51:52.873135 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889397c4-85f2-41d2-8739-0f3591c12e03-tls-certificates podName:889397c4-85f2-41d2-8739-0f3591c12e03 nodeName:}" failed. No retries permitted until 2026-07-23 18:51:53.373119231 +0000 UTC m=+65.954263429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/889397c4-85f2-41d2-8739-0f3591c12e03-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-26rst" (UID: "889397c4-85f2-41d2-8739-0f3591c12e03") : secret "prometheus-operator-admission-webhook-tls" not found Jul 23 18:51:53.050121 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.050047 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:53.376615 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.376588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/889397c4-85f2-41d2-8739-0f3591c12e03-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-26rst\" (UID: \"889397c4-85f2-41d2-8739-0f3591c12e03\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:53.379144 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.379124 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/889397c4-85f2-41d2-8739-0f3591c12e03-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-26rst\" (UID: \"889397c4-85f2-41d2-8739-0f3591c12e03\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:53.590004 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.589965 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:53.678328 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.678293 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:53.681232 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.681208 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 18:51:53.691619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.691598 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfb1861c-a4c3-4224-87ba-df1e74c3df08-metrics-certs\") pod \"network-metrics-daemon-zbwbh\" (UID: \"bfb1861c-a4c3-4224-87ba-df1e74c3df08\") " pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:53.711876 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.711851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst"] Jul 23 18:51:53.714709 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:53.714687 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889397c4_85f2_41d2_8739_0f3591c12e03.slice/crio-9af30a7cf94e56f9e2f8ae8bbe646ee7686332f9159c0a2f1960bdeabdec7276 WatchSource:0}: Error finding container 9af30a7cf94e56f9e2f8ae8bbe646ee7686332f9159c0a2f1960bdeabdec7276: Status 404 returned error can't find the container with id 9af30a7cf94e56f9e2f8ae8bbe646ee7686332f9159c0a2f1960bdeabdec7276 Jul 23 18:51:53.778809 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.778773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:53.778996 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.778813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:53.781367 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.781351 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 18:51:53.784289 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.784275 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 18:51:53.791747 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.791729 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a943505d-6390-4fe2-b647-70f281a4d58a-original-pull-secret\") pod \"global-pull-secret-syncer-wphxg\" (UID: \"a943505d-6390-4fe2-b647-70f281a4d58a\") " pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:53.793412 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.793398 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 18:51:53.801912 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.801897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k88ps\" (UniqueName: \"kubernetes.io/projected/63e75e39-7a09-4754-bf45-8d687b92ee09-kube-api-access-k88ps\") pod \"network-check-target-wkn9h\" (UID: \"63e75e39-7a09-4754-bf45-8d687b92ee09\") " pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:53.920841 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.920754 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c83006d-4a76-4def-96fe-755b5a6da28b" path="/var/lib/kubelet/pods/8c83006d-4a76-4def-96fe-755b5a6da28b/volumes" Jul 23 18:51:53.928349 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.928332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wphxg" Jul 23 18:51:53.938067 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.938049 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8ssn\"" Jul 23 18:51:53.942237 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.942216 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mqhg9\"" Jul 23 18:51:53.946467 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.946451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbwbh" Jul 23 18:51:53.950699 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:53.950680 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:54.075691 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.075602 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wphxg"] Jul 23 18:51:54.077896 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:54.077869 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda943505d_6390_4fe2_b647_70f281a4d58a.slice/crio-f9718f63090b7e4a73514291e3a5e8a24515cb52696e3c97be2aeb58752dd3c2 WatchSource:0}: Error finding container f9718f63090b7e4a73514291e3a5e8a24515cb52696e3c97be2aeb58752dd3c2: Status 404 returned error can't find the container with id f9718f63090b7e4a73514291e3a5e8a24515cb52696e3c97be2aeb58752dd3c2 Jul 23 18:51:54.113881 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.113861 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zbwbh"] Jul 23 18:51:54.115446 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:54.115424 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb1861c_a4c3_4224_87ba_df1e74c3df08.slice/crio-53073681b3493bd0149e317d625364343b9e889403638f38e60e8b4321c27823 WatchSource:0}: Error finding container 53073681b3493bd0149e317d625364343b9e889403638f38e60e8b4321c27823: Status 404 returned error can't find the container with id 53073681b3493bd0149e317d625364343b9e889403638f38e60e8b4321c27823 Jul 23 18:51:54.134612 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.134587 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wkn9h"] Jul 23 18:51:54.136969 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:54.136945 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e75e39_7a09_4754_bf45_8d687b92ee09.slice/crio-d2193abcf79cfabed552453a9e9b81354cd7fd00ed0b0a37918b0d6c771df0ac WatchSource:0}: Error finding container d2193abcf79cfabed552453a9e9b81354cd7fd00ed0b0a37918b0d6c771df0ac: Status 404 returned error can't find the container with id d2193abcf79cfabed552453a9e9b81354cd7fd00ed0b0a37918b0d6c771df0ac Jul 23 18:51:54.305708 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.305606 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wphxg" event={"ID":"a943505d-6390-4fe2-b647-70f281a4d58a","Type":"ContainerStarted","Data":"f9718f63090b7e4a73514291e3a5e8a24515cb52696e3c97be2aeb58752dd3c2"} Jul 23 18:51:54.307110 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.307057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wkn9h" event={"ID":"63e75e39-7a09-4754-bf45-8d687b92ee09","Type":"ContainerStarted","Data":"d2193abcf79cfabed552453a9e9b81354cd7fd00ed0b0a37918b0d6c771df0ac"} Jul 23 18:51:54.308223 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.308196 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" event={"ID":"889397c4-85f2-41d2-8739-0f3591c12e03","Type":"ContainerStarted","Data":"9af30a7cf94e56f9e2f8ae8bbe646ee7686332f9159c0a2f1960bdeabdec7276"} Jul 23 18:51:54.309338 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.309313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zbwbh" event={"ID":"bfb1861c-a4c3-4224-87ba-df1e74c3df08","Type":"ContainerStarted","Data":"53073681b3493bd0149e317d625364343b9e889403638f38e60e8b4321c27823"} Jul 23 18:51:54.762097 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.762064 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:54.860751 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:54.860468 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:55.316038 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.316009 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" event={"ID":"889397c4-85f2-41d2-8739-0f3591c12e03","Type":"ContainerStarted","Data":"2284af1e39fd1e48fdfefd6942a4edd9c9e73d43e5e276404474c88de00b9899"} Jul 23 18:51:55.316712 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.316690 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:55.324337 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.324246 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" Jul 23 18:51:55.341262 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.341169 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-26rst" podStartSLOduration=2.279551066 podStartE2EDuration="3.341149652s" totalImagesPullingTime="1.061598586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:55.338764241 +0000 UTC m=+67.919908462" watchObservedRunningTime="2026-07-23 18:51:55.341149652 +0000 UTC m=+67.922293892" Jul 23 18:51:55.744813 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.744772 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:55.797261 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.797220 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv"] Jul 23 18:51:55.820691 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.819137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.820948 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.820914 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv"] Jul 23 18:51:55.821971 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.821950 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 18:51:55.823584 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.823563 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 18:51:55.823584 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.823578 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 18:51:55.824429 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.824402 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gczbp\"" Jul 23 18:51:55.824542 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.824456 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 18:51:55.824542 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.824504 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 18:51:55.853195 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.853160 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:55.894198 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.894167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ad01d67-ceac-45a6-b473-e54f34910705-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.894198 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.894201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ad01d67-ceac-45a6-b473-e54f34910705-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.894424 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.894223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9gk\" (UniqueName: \"kubernetes.io/projected/1ad01d67-ceac-45a6-b473-e54f34910705-kube-api-access-cc9gk\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.894424 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.894312 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad01d67-ceac-45a6-b473-e54f34910705-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.928203 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.928164 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:55.995470 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.995390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad01d67-ceac-45a6-b473-e54f34910705-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.995470 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.995461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ad01d67-ceac-45a6-b473-e54f34910705-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.995695 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.995488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ad01d67-ceac-45a6-b473-e54f34910705-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.995695 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.995511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9gk\" (UniqueName: \"kubernetes.io/projected/1ad01d67-ceac-45a6-b473-e54f34910705-kube-api-access-cc9gk\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.996243 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.996197 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad01d67-ceac-45a6-b473-e54f34910705-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.998384 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.998356 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ad01d67-ceac-45a6-b473-e54f34910705-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:55.998491 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:55.998386 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ad01d67-ceac-45a6-b473-e54f34910705-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:56.010912 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.010893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9gk\" (UniqueName: \"kubernetes.io/projected/1ad01d67-ceac-45a6-b473-e54f34910705-kube-api-access-cc9gk\") pod \"prometheus-operator-5945b5cfd-z5qxv\" (UID: \"1ad01d67-ceac-45a6-b473-e54f34910705\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:56.133882 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.133849 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" Jul 23 18:51:56.323600 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.323564 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zbwbh" event={"ID":"bfb1861c-a4c3-4224-87ba-df1e74c3df08","Type":"ContainerStarted","Data":"7821f030794f02a56c023c6d2123fcad47046a6899a17675c6f0c7af8afcdf5e"} Jul 23 18:51:56.323991 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.323631 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv"] Jul 23 18:51:56.323991 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.323667 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zbwbh" event={"ID":"bfb1861c-a4c3-4224-87ba-df1e74c3df08","Type":"ContainerStarted","Data":"679cfa312d43c95be4d61c8a847c270cdf1434b66b8a08ee64487a0c1036d9c9"} Jul 23 18:51:56.327912 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:51:56.327886 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad01d67_ceac_45a6_b473_e54f34910705.slice/crio-89cce5138d0870c1a6d0c61f49ce8bac0171b2a067db5d030fe5dcb0733b6a5b WatchSource:0}: Error finding container 89cce5138d0870c1a6d0c61f49ce8bac0171b2a067db5d030fe5dcb0733b6a5b: Status 404 returned error can't find the container with id 89cce5138d0870c1a6d0c61f49ce8bac0171b2a067db5d030fe5dcb0733b6a5b Jul 23 18:51:56.511525 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.511494 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:56.610944 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.610867 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:56.716411 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.716354 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:56.813453 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:56.813418 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:57.328695 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:57.328653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" event={"ID":"1ad01d67-ceac-45a6-b473-e54f34910705","Type":"ContainerStarted","Data":"89cce5138d0870c1a6d0c61f49ce8bac0171b2a067db5d030fe5dcb0733b6a5b"} Jul 23 18:51:58.425998 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:58.425967 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:58.560163 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:58.560133 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:58.693494 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:58.693319 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:58.913171 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:58.913134 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:59.129214 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.129178 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:59.337681 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.337645 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wkn9h" event={"ID":"63e75e39-7a09-4754-bf45-8d687b92ee09","Type":"ContainerStarted","Data":"fa7e2fa5cda1b608f56c337132338ba00b3dcc8881f9991bc800f55355200cad"} Jul 23 18:51:59.337881 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.337760 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:51:59.339455 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.339417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wphxg" event={"ID":"a943505d-6390-4fe2-b647-70f281a4d58a","Type":"ContainerStarted","Data":"bcbf6dde4a4ab91bc1c3b85989291ce3b413505d5760007140c1018f80c7c59b"} Jul 23 18:51:59.365217 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.365179 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:59.370151 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.369923 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zbwbh" podStartSLOduration=70.218113783 podStartE2EDuration="1m11.369906289s" totalImagesPullingTime="1.151792506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:56.353901339 +0000 UTC m=+68.935045560" watchObservedRunningTime="2026-07-23 18:51:59.369906289 +0000 UTC m=+71.951050513" Jul 23 18:51:59.370650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.370616 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wkn9h" podStartSLOduration=66.480465416 podStartE2EDuration="1m11.370609218s" totalImagesPullingTime="4.890143802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:59.37005279 +0000 UTC m=+71.951197011" watchObservedRunningTime="2026-07-23 18:51:59.370609218 +0000 UTC m=+71.951753437" Jul 23 18:51:59.412519 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.412469 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wphxg" podStartSLOduration=66.460553524 podStartE2EDuration="1m11.412449514s" totalImagesPullingTime="4.95189599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:51:59.412291462 +0000 UTC m=+71.993435684" watchObservedRunningTime="2026-07-23 18:51:59.412449514 +0000 UTC m=+71.993593735" Jul 23 18:51:59.514243 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:51:59.514198 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:00.190379 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.190348 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:00.345346 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.345311 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" event={"ID":"1ad01d67-ceac-45a6-b473-e54f34910705","Type":"ContainerStarted","Data":"2bbf49708abbd7dae0e922e73aa47315080670233bf3e09d96b0f5d7725ef9f1"} Jul 23 18:52:00.345519 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.345369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" event={"ID":"1ad01d67-ceac-45a6-b473-e54f34910705","Type":"ContainerStarted","Data":"340f536910ef025571b2ef86ed6f82f07c6ad84ae4ca1396d63f9ec81f3a97c9"} Jul 23 18:52:00.386803 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.386771 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:00.391503 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.391456 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z5qxv" podStartSLOduration=1.9248230450000001 podStartE2EDuration="5.391441389s" totalImagesPullingTime="3.466618344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:00.390949467 +0000 UTC m=+72.972093687" watchObservedRunningTime="2026-07-23 18:52:00.391441389 +0000 UTC m=+72.972585609" Jul 23 18:52:00.575262 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.575230 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:00.796445 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.796415 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:00.902281 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:00.902209 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.020839 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.020787 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.154726 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.154694 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.341578 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.341541 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.393394 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.393364 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq"] Jul 23 18:52:02.398358 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.398330 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.405815 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.405795 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 23 18:52:02.405960 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.405870 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-d5zjq\"" Jul 23 18:52:02.405960 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.405875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 23 18:52:02.424686 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.424649 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq"] Jul 23 18:52:02.448043 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.448013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.448179 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.448052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.448179 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.448079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.448179 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.448109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67r6k\" (UniqueName: \"kubernetes.io/projected/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-kube-api-access-67r6k\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.483810 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.483779 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z2g9t"] Jul 23 18:52:02.485958 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.485940 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.488973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.488949 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 18:52:02.489254 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.489241 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-csbv8\"" Jul 23 18:52:02.489559 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.489539 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 18:52:02.489634 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.489539 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 18:52:02.519763 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.519736 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.548842 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.548972 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.548972 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-sys\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.548972 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548931 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.548972 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-wtmp\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm6c2\" (UniqueName: \"kubernetes.io/projected/845b068d-c7f2-401f-9213-528263a4bee5-kube-api-access-pm6c2\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.548994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-root\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/845b068d-c7f2-401f-9213-528263a4bee5-metrics-client-ca\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549033 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-textfile\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.549124 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-tls\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.549442 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67r6k\" (UniqueName: \"kubernetes.io/projected/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-kube-api-access-67r6k\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.549583 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.549565 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.551541 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.551519 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.551628 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.551523 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.569215 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.569187 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67r6k\" (UniqueName: \"kubernetes.io/projected/49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4-kube-api-access-67r6k\") pod \"openshift-state-metrics-79f6958cdf-fdmbq\" (UID: \"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.614869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.614769 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.649795 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-tls\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.649952 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.649952 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.649952 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649933 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-sys\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-wtmp\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm6c2\" (UniqueName: \"kubernetes.io/projected/845b068d-c7f2-401f-9213-528263a4bee5-kube-api-access-pm6c2\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.649995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-root\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/845b068d-c7f2-401f-9213-528263a4bee5-metrics-client-ca\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-textfile\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650070 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-sys\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650354 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-root\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650354 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650146 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-wtmp\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650354 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:52:02.650213 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 18:52:02.650354 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:52:02.650292 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-tls podName:845b068d-c7f2-401f-9213-528263a4bee5 nodeName:}" failed. No retries permitted until 2026-07-23 18:52:03.150270092 +0000 UTC m=+75.731414305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-tls") pod "node-exporter-z2g9t" (UID: "845b068d-c7f2-401f-9213-528263a4bee5") : secret "node-exporter-tls" not found Jul 23 18:52:02.650525 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-textfile\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650572 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650537 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.650607 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.650598 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/845b068d-c7f2-401f-9213-528263a4bee5-metrics-client-ca\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.652219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.652201 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.662954 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.662926 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm6c2\" (UniqueName: \"kubernetes.io/projected/845b068d-c7f2-401f-9213-528263a4bee5-kube-api-access-pm6c2\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:02.707066 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.707036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" Jul 23 18:52:02.780757 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.780726 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:02.869497 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:02.869475 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq"] Jul 23 18:52:02.871229 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:52:02.871206 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f2dc49_4fc1_42a9_8fa2_bb18d9ce3ab4.slice/crio-e71b54cac69cb448fe0a6803079e84909900ec42b45bc90a99d6b73bcbc636ab WatchSource:0}: Error finding container e71b54cac69cb448fe0a6803079e84909900ec42b45bc90a99d6b73bcbc636ab: Status 404 returned error can't find the container with id e71b54cac69cb448fe0a6803079e84909900ec42b45bc90a99d6b73bcbc636ab Jul 23 18:52:03.038447 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.038390 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.154524 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.154433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-tls\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:03.156946 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.156924 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/845b068d-c7f2-401f-9213-528263a4bee5-node-exporter-tls\") pod \"node-exporter-z2g9t\" (UID: \"845b068d-c7f2-401f-9213-528263a4bee5\") " pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:03.325617 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.325562 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.357129 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.357099 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" event={"ID":"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4","Type":"ContainerStarted","Data":"fddb8f26e804c63ec927b0295617e4eef97b0f802e031d168daa1947814e60d6"} Jul 23 18:52:03.357298 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.357154 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" event={"ID":"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4","Type":"ContainerStarted","Data":"be7d1d1faf11716cc01c4828b47c58ff5fcc4dae209fee09c51f210a8405bf28"} Jul 23 18:52:03.357298 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.357174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" event={"ID":"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4","Type":"ContainerStarted","Data":"e71b54cac69cb448fe0a6803079e84909900ec42b45bc90a99d6b73bcbc636ab"} Jul 23 18:52:03.394800 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.394769 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z2g9t" Jul 23 18:52:03.403976 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:52:03.403953 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845b068d_c7f2_401f_9213_528263a4bee5.slice/crio-3fe3584bd6dafa57862d092f2a1fb9eaa6ac96b41aa4cd97e3bd42e76fd157c4 WatchSource:0}: Error finding container 3fe3584bd6dafa57862d092f2a1fb9eaa6ac96b41aa4cd97e3bd42e76fd157c4: Status 404 returned error can't find the container with id 3fe3584bd6dafa57862d092f2a1fb9eaa6ac96b41aa4cd97e3bd42e76fd157c4 Jul 23 18:52:03.444185 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.444159 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.704679 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.704598 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.898354 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.898319 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c8475c8c8-svnhp"] Jul 23 18:52:03.900771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.900749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:03.905646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.905627 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 18:52:03.905946 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.905928 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 18:52:03.906602 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.906582 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 18:52:03.906705 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.906674 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 18:52:03.906905 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.906888 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 18:52:03.907166 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.907151 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wqc6d\"" Jul 23 18:52:03.907247 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.907192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 18:52:03.911276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.911261 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 18:52:03.929518 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.929493 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8475c8c8-svnhp"] Jul 23 18:52:03.950105 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.950064 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.959955 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.959891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-oauth-config\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:03.959955 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.959940 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-serving-cert\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:03.960126 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.959969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-427qt\" (UniqueName: \"kubernetes.io/projected/99d8759f-b8dc-461f-8e67-df6243808a90-kube-api-access-427qt\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:03.960126 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.959994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-service-ca\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:03.960126 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.960077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-console-config\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:03.960126 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:03.960122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-oauth-serving-cert\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.060678 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.060655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-427qt\" (UniqueName: \"kubernetes.io/projected/99d8759f-b8dc-461f-8e67-df6243808a90-kube-api-access-427qt\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.060686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-service-ca\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.060724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-console-config\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.060755 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-oauth-serving-cert\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.060787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-oauth-config\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.060835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-serving-cert\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061477 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.061451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-service-ca\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061601 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.061571 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-oauth-serving-cert\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.061772 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.061751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-console-config\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.064858 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.064159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-oauth-config\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.064858 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.064174 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-serving-cert\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.072783 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.072764 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-427qt\" (UniqueName: \"kubernetes.io/projected/99d8759f-b8dc-461f-8e67-df6243808a90-kube-api-access-427qt\") pod \"console-c8475c8c8-svnhp\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.211747 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.211659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:04.362803 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.362765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" event={"ID":"49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4","Type":"ContainerStarted","Data":"8bd7e344942cf4bde92ea8ff27da0f1368d46a9a7d666907995ee948bad476ca"} Jul 23 18:52:04.364462 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.364436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2g9t" event={"ID":"845b068d-c7f2-401f-9213-528263a4bee5","Type":"ContainerStarted","Data":"3fe3584bd6dafa57862d092f2a1fb9eaa6ac96b41aa4cd97e3bd42e76fd157c4"} Jul 23 18:52:04.366509 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.366485 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8475c8c8-svnhp"] Jul 23 18:52:04.492582 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:52:04.492511 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d8759f_b8dc_461f_8e67_df6243808a90.slice/crio-bf51a855f1148d9592f8946db8e6228d5e12ebed543f50cbdbeacbea8591f4c5 WatchSource:0}: Error finding container bf51a855f1148d9592f8946db8e6228d5e12ebed543f50cbdbeacbea8591f4c5: Status 404 returned error can't find the container with id bf51a855f1148d9592f8946db8e6228d5e12ebed543f50cbdbeacbea8591f4c5 Jul 23 18:52:04.737615 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.737584 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:04.879778 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:04.879748 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:05.376242 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.376207 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8475c8c8-svnhp" event={"ID":"99d8759f-b8dc-461f-8e67-df6243808a90","Type":"ContainerStarted","Data":"bf51a855f1148d9592f8946db8e6228d5e12ebed543f50cbdbeacbea8591f4c5"} Jul 23 18:52:05.378925 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.378895 2578 generic.go:356] "Generic (PLEG): container finished" podID="845b068d-c7f2-401f-9213-528263a4bee5" containerID="2b5f683db66e75d078aa2c8039660712c56d5ff216dc21ffed723c2128bb6831" exitCode=0 Jul 23 18:52:05.379105 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.379075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2g9t" event={"ID":"845b068d-c7f2-401f-9213-528263a4bee5","Type":"ContainerDied","Data":"2b5f683db66e75d078aa2c8039660712c56d5ff216dc21ffed723c2128bb6831"} Jul 23 18:52:05.471171 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.471130 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-fdmbq" podStartSLOduration=2.499473126 podStartE2EDuration="3.471114476s" totalImagesPullingTime="971.64135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:04.401945169 +0000 UTC m=+76.983089389" watchObservedRunningTime="2026-07-23 18:52:05.471114476 +0000 UTC m=+78.052258696" Jul 23 18:52:05.671680 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.671602 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-599ff9bf95-fjqv6"] Jul 23 18:52:05.675380 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.675359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.678681 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.678660 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lptb9\"" Jul 23 18:52:05.678783 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.678749 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 18:52:05.678926 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.678908 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bi7et5j1oighv\"" Jul 23 18:52:05.679306 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.679270 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 18:52:05.679306 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.679306 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 18:52:05.679471 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.679365 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 18:52:05.679471 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.679385 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 18:52:05.708017 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.707992 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-599ff9bf95-fjqv6"] Jul 23 18:52:05.775537 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775505 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7866138c-9592-4ed3-89a5-df4363514b42-metrics-client-ca\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775683 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-grpc-tls\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775683 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775683 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-tls\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr4zl\" (UniqueName: \"kubernetes.io/projected/7866138c-9592-4ed3-89a5-df4363514b42-kube-api-access-qr4zl\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.775869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.775865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.798443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.798409 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:05.877017 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.876981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-grpc-tls\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877017 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877178 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877311 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877348 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-tls\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877386 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr4zl\" (UniqueName: \"kubernetes.io/projected/7866138c-9592-4ed3-89a5-df4363514b42-kube-api-access-qr4zl\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877447 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.877497 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.877476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7866138c-9592-4ed3-89a5-df4363514b42-metrics-client-ca\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.878426 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.878395 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7866138c-9592-4ed3-89a5-df4363514b42-metrics-client-ca\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.880269 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.880245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.880385 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.880279 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.880385 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.880306 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.880502 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.880477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-grpc-tls\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.880803 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.880775 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-tls\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.881473 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.881451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7866138c-9592-4ed3-89a5-df4363514b42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.892746 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.892711 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr4zl\" (UniqueName: \"kubernetes.io/projected/7866138c-9592-4ed3-89a5-df4363514b42-kube-api-access-qr4zl\") pod \"thanos-querier-599ff9bf95-fjqv6\" (UID: \"7866138c-9592-4ed3-89a5-df4363514b42\") " pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:05.925214 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.925134 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:05.986163 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:05.986133 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:06.094786 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.094757 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:06.161111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.161086 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-599ff9bf95-fjqv6"] Jul 23 18:52:06.161690 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:52:06.161663 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7866138c_9592_4ed3_89a5_df4363514b42.slice/crio-55af74dbea960641bfe61b5e9023ec3522f652e3af8e53cc496479c324880b9f WatchSource:0}: Error finding container 55af74dbea960641bfe61b5e9023ec3522f652e3af8e53cc496479c324880b9f: Status 404 returned error can't find the container with id 55af74dbea960641bfe61b5e9023ec3522f652e3af8e53cc496479c324880b9f Jul 23 18:52:06.311448 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.311365 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:06.389277 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.389242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2g9t" event={"ID":"845b068d-c7f2-401f-9213-528263a4bee5","Type":"ContainerStarted","Data":"e29f8c26c5ef1a483c86f2b7508449a80f4a2def7eadb829fa27e954cd080b7d"} Jul 23 18:52:06.389686 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.389295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2g9t" event={"ID":"845b068d-c7f2-401f-9213-528263a4bee5","Type":"ContainerStarted","Data":"d6f45492885d9d26e053382d2f6d9178735434e08223b1b603f2670707ea1235"} Jul 23 18:52:06.390925 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.390896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"55af74dbea960641bfe61b5e9023ec3522f652e3af8e53cc496479c324880b9f"} Jul 23 18:52:06.444950 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:06.444905 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z2g9t" podStartSLOduration=3.240653212 podStartE2EDuration="4.444889259s" totalImagesPullingTime="1.135255067s" totalInitContainerRuntime="68.98098ms" isStatefulPod=true podCreationTimestamp="2026-07-23 18:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:06.443921408 +0000 UTC m=+79.025065629" watchObservedRunningTime="2026-07-23 18:52:06.444889259 +0000 UTC m=+79.026033480" Jul 23 18:52:07.044938 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.044893 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:07.080670 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.080640 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777"] Jul 23 18:52:07.089241 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.089216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.093262 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.093237 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wppnk\"" Jul 23 18:52:07.093871 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.093851 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 18:52:07.113059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.112988 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777"] Jul 23 18:52:07.186871 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.186833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.187052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.186969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s45c7\" (UniqueName: \"kubernetes.io/projected/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-kube-api-access-s45c7\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.201267 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.201200 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:07.287932 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.287899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.288147 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.287970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45c7\" (UniqueName: \"kubernetes.io/projected/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-kube-api-access-s45c7\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.288147 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:52:07.288019 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 23 18:52:07.288147 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:52:07.288090 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-monitoring-plugin-cert podName:f402f241-1c8f-4057-a3a4-212d5bf6bf5e nodeName:}" failed. No retries permitted until 2026-07-23 18:52:07.788074859 +0000 UTC m=+80.369219062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-hm777" (UID: "f402f241-1c8f-4057-a3a4-212d5bf6bf5e") : secret "monitoring-plugin-cert" not found Jul 23 18:52:07.308303 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.308237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s45c7\" (UniqueName: \"kubernetes.io/projected/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-kube-api-access-s45c7\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.419972 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.419903 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:07.792997 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.792966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:07.795905 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:07.795868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f402f241-1c8f-4057-a3a4-212d5bf6bf5e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hm777\" (UID: \"f402f241-1c8f-4057-a3a4-212d5bf6bf5e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:08.000165 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.000137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:08.150129 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.150101 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777"] Jul 23 18:52:08.153101 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:52:08.153076 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf402f241_1c8f_4057_a3a4_212d5bf6bf5e.slice/crio-108d87ed54f25bcd7af4609c0bf0e778249cd05fad70c4d711f26d2917819d29 WatchSource:0}: Error finding container 108d87ed54f25bcd7af4609c0bf0e778249cd05fad70c4d711f26d2917819d29: Status 404 returned error can't find the container with id 108d87ed54f25bcd7af4609c0bf0e778249cd05fad70c4d711f26d2917819d29 Jul 23 18:52:08.405421 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.405345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8475c8c8-svnhp" event={"ID":"99d8759f-b8dc-461f-8e67-df6243808a90","Type":"ContainerStarted","Data":"b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68"} Jul 23 18:52:08.406900 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.406875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" event={"ID":"f402f241-1c8f-4057-a3a4-212d5bf6bf5e","Type":"ContainerStarted","Data":"108d87ed54f25bcd7af4609c0bf0e778249cd05fad70c4d711f26d2917819d29"} Jul 23 18:52:08.435803 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.435321 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c8475c8c8-svnhp" podStartSLOduration=2.581582374 podStartE2EDuration="5.435299527s" totalImagesPullingTime="2.853717153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:08.43391686 +0000 UTC m=+81.015061082" watchObservedRunningTime="2026-07-23 18:52:08.435299527 +0000 UTC m=+81.016443750" Jul 23 18:52:08.579958 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.579814 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:08.664569 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:08.664491 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:09.270510 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:09.270416 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:09.413891 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:09.413857 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"e13176530a2be726d3536a155aadc3358068f868be586f8f1ea44ecd5e5f9595"} Jul 23 18:52:09.414050 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:09.413942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"24385f5bdc6945abde806985308f86608bc6b596a69031d507bbee94ca1369ab"} Jul 23 18:52:09.414050 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:09.413988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"cb0f9cb6d9ebcf92e8ea90d280616410a74c3331ead4a8f432fa7b77a6805758"} Jul 23 18:52:10.419353 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:10.419316 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" event={"ID":"f402f241-1c8f-4057-a3a4-212d5bf6bf5e","Type":"ContainerStarted","Data":"bbe30ea940962f3d56f07611460f22d4a736ec233425479b0584838294f8e304"} Jul 23 18:52:10.419802 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:10.419447 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:10.424138 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:10.424117 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" Jul 23 18:52:10.453268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:10.453230 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hm777" podStartSLOduration=1.67422286 podStartE2EDuration="3.45321962s" totalImagesPullingTime="1.77899676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:10.452701756 +0000 UTC m=+83.033845976" watchObservedRunningTime="2026-07-23 18:52:10.45321962 +0000 UTC m=+83.034363840" Jul 23 18:52:11.426533 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:11.426497 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"3770135a9c223772c66e550783304e29a56a9c5c33acde03126eaa369870603f"} Jul 23 18:52:11.426940 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:11.426572 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"c42caa1fb9d90df7b94ec00cb6b15ff00106d259c5e37646d8e9b5e18a29b76c"} Jul 23 18:52:11.426940 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:11.426603 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" event={"ID":"7866138c-9592-4ed3-89a5-df4363514b42","Type":"ContainerStarted","Data":"c7b2baec6ce4d8124dcf413475354aade7e458c6cc8ea2ddcb4a9b7e2ba7c352"} Jul 23 18:52:11.511201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:11.511157 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" podStartSLOduration=2.297496324 podStartE2EDuration="6.511141882s" totalImagesPullingTime="4.213645558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:52:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:11.510753278 +0000 UTC m=+84.091897498" watchObservedRunningTime="2026-07-23 18:52:11.511141882 +0000 UTC m=+84.092286146" Jul 23 18:52:12.430310 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:12.430279 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:13.522119 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.522081 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:13.629057 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.629026 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:13.683806 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.683780 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cb8c85cc4-4pdv6"] Jul 23 18:52:13.731646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.731618 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:13.802883 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.802796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb8c85cc4-4pdv6"] Jul 23 18:52:13.803009 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.802928 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.811206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.811185 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 18:52:13.845232 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.845208 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:13.942300 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942273 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsppr\" (UniqueName: \"kubernetes.io/projected/759e47d5-b00f-49cb-908a-be9005620df6-kube-api-access-nsppr\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.942438 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-service-ca\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.942438 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-trusted-ca-bundle\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.942438 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-oauth-serving-cert\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.942438 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-console-config\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.942593 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-serving-cert\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:13.942593 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:13.942463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-oauth-config\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.043600 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-service-ca\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.043750 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-trusted-ca-bundle\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.043750 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-oauth-serving-cert\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.043857 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-console-config\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.043857 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-serving-cert\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.043857 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-oauth-config\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.044004 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.043892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsppr\" (UniqueName: \"kubernetes.io/projected/759e47d5-b00f-49cb-908a-be9005620df6-kube-api-access-nsppr\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.044415 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.044385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-oauth-serving-cert\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.044523 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.044472 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-service-ca\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.044523 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.044489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-console-config\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.044796 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.044622 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-trusted-ca-bundle\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.046413 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.046391 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-oauth-config\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.046494 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.046473 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-serving-cert\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.053361 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.053302 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsppr\" (UniqueName: \"kubernetes.io/projected/759e47d5-b00f-49cb-908a-be9005620df6-kube-api-access-nsppr\") pod \"console-7cb8c85cc4-4pdv6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.112857 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.112815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:14.212565 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.212537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:14.212692 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.212611 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:14.219200 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.218989 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:14.294442 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.294313 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb8c85cc4-4pdv6"] Jul 23 18:52:14.297038 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:52:14.297014 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759e47d5_b00f_49cb_908a_be9005620df6.slice/crio-6ddcd2548448e73a5f6008af02f578309ee3e7456d8f507b07b4e0b23a8ef08c WatchSource:0}: Error finding container 6ddcd2548448e73a5f6008af02f578309ee3e7456d8f507b07b4e0b23a8ef08c: Status 404 returned error can't find the container with id 6ddcd2548448e73a5f6008af02f578309ee3e7456d8f507b07b4e0b23a8ef08c Jul 23 18:52:14.440551 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.440515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb8c85cc4-4pdv6" event={"ID":"759e47d5-b00f-49cb-908a-be9005620df6","Type":"ContainerStarted","Data":"188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009"} Jul 23 18:52:14.440740 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.440571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb8c85cc4-4pdv6" event={"ID":"759e47d5-b00f-49cb-908a-be9005620df6","Type":"ContainerStarted","Data":"6ddcd2548448e73a5f6008af02f578309ee3e7456d8f507b07b4e0b23a8ef08c"} Jul 23 18:52:14.444470 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.444450 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:14.475767 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:14.475721 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cb8c85cc4-4pdv6" podStartSLOduration=1.475703763 podStartE2EDuration="1.475703763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:52:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:52:14.474859058 +0000 UTC m=+87.056003275" watchObservedRunningTime="2026-07-23 18:52:14.475703763 +0000 UTC m=+87.056847983" Jul 23 18:52:17.616709 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:17.616674 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:17.716271 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:17.716233 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:17.853652 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:17.853611 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:18.442718 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:18.442690 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-599ff9bf95-fjqv6" Jul 23 18:52:19.455250 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:19.455215 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:19.594657 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:19.594625 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:23.282302 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:23.282266 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:23.399636 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:23.399603 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:24.113714 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:24.113676 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:24.113940 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:24.113738 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:24.118423 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:24.118400 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:24.480400 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:24.480329 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:52:24.567035 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:24.567005 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8475c8c8-svnhp"] Jul 23 18:52:25.722980 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:25.722944 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:25.853831 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:25.853801 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:27.120684 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:27.120650 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:27.251171 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:27.251136 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:27.989265 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:27.989231 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:28.140912 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:28.140874 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:28.913511 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:28.913474 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:29.019989 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:29.019951 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:30.347904 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:30.347871 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wkn9h" Jul 23 18:52:35.267453 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:35.267412 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:35.481878 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:35.481845 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:35.621423 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:35.621390 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:35.732306 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:35.732273 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:42.957369 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:42.957328 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:43.059115 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:43.059089 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:49.586943 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.586890 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c8475c8c8-svnhp" podUID="99d8759f-b8dc-461f-8e67-df6243808a90" containerName="console" containerID="cri-o://b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68" gracePeriod=15 Jul 23 18:52:49.827607 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.827582 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:49.919714 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.919629 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-console-config\") pod \"99d8759f-b8dc-461f-8e67-df6243808a90\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " Jul 23 18:52:49.919714 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.919667 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-serving-cert\") pod \"99d8759f-b8dc-461f-8e67-df6243808a90\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " Jul 23 18:52:49.919714 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.919712 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-oauth-config\") pod \"99d8759f-b8dc-461f-8e67-df6243808a90\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " Jul 23 18:52:49.920016 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.919732 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-service-ca\") pod \"99d8759f-b8dc-461f-8e67-df6243808a90\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " Jul 23 18:52:49.920016 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.919758 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-427qt\" (UniqueName: \"kubernetes.io/projected/99d8759f-b8dc-461f-8e67-df6243808a90-kube-api-access-427qt\") pod \"99d8759f-b8dc-461f-8e67-df6243808a90\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " Jul 23 18:52:49.920016 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.919799 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-oauth-serving-cert\") pod \"99d8759f-b8dc-461f-8e67-df6243808a90\" (UID: \"99d8759f-b8dc-461f-8e67-df6243808a90\") " Jul 23 18:52:49.920219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.920191 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-service-ca" (OuterVolumeSpecName: "service-ca") pod "99d8759f-b8dc-461f-8e67-df6243808a90" (UID: "99d8759f-b8dc-461f-8e67-df6243808a90"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:52:49.920341 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.920242 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "99d8759f-b8dc-461f-8e67-df6243808a90" (UID: "99d8759f-b8dc-461f-8e67-df6243808a90"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:52:49.920341 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.920317 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-console-config" (OuterVolumeSpecName: "console-config") pod "99d8759f-b8dc-461f-8e67-df6243808a90" (UID: "99d8759f-b8dc-461f-8e67-df6243808a90"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:52:49.922274 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.922247 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "99d8759f-b8dc-461f-8e67-df6243808a90" (UID: "99d8759f-b8dc-461f-8e67-df6243808a90"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:52:49.922482 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.922460 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "99d8759f-b8dc-461f-8e67-df6243808a90" (UID: "99d8759f-b8dc-461f-8e67-df6243808a90"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:52:49.922527 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:49.922477 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d8759f-b8dc-461f-8e67-df6243808a90-kube-api-access-427qt" (OuterVolumeSpecName: "kube-api-access-427qt") pod "99d8759f-b8dc-461f-8e67-df6243808a90" (UID: "99d8759f-b8dc-461f-8e67-df6243808a90"). InnerVolumeSpecName "kube-api-access-427qt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:50.021115 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.021079 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-oauth-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:52:50.021115 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.021111 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:52:50.021115 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.021120 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-427qt\" (UniqueName: \"kubernetes.io/projected/99d8759f-b8dc-461f-8e67-df6243808a90-kube-api-access-427qt\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:52:50.021342 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.021129 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-oauth-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:52:50.021342 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.021137 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d8759f-b8dc-461f-8e67-df6243808a90-console-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:52:50.021342 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.021146 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d8759f-b8dc-461f-8e67-df6243808a90-console-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:52:50.263635 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.263555 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:50.448186 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.448154 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:50.568646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.568562 2578 generic.go:356] "Generic (PLEG): container finished" podID="99d8759f-b8dc-461f-8e67-df6243808a90" containerID="b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68" exitCode=0 Jul 23 18:52:50.568646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.568606 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8475c8c8-svnhp" Jul 23 18:52:50.568646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.568616 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8475c8c8-svnhp" event={"ID":"99d8759f-b8dc-461f-8e67-df6243808a90","Type":"ContainerDied","Data":"b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68"} Jul 23 18:52:50.568881 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.568659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8475c8c8-svnhp" event={"ID":"99d8759f-b8dc-461f-8e67-df6243808a90","Type":"ContainerDied","Data":"bf51a855f1148d9592f8946db8e6228d5e12ebed543f50cbdbeacbea8591f4c5"} Jul 23 18:52:50.568881 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.568682 2578 scope.go:117] "RemoveContainer" containerID="b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68" Jul 23 18:52:50.576974 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.576960 2578 scope.go:117] "RemoveContainer" containerID="b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68" Jul 23 18:52:50.577222 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:52:50.577202 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68\": container with ID starting with b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68 not found: ID does not exist" containerID="b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68" Jul 23 18:52:50.577268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.577231 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68"} err="failed to get container status \"b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68\": rpc error: code = NotFound desc = could not find container \"b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68\": container with ID starting with b778a8c7c4596d58db9aebf6d855eb2e7d8f2bb24e41a3cbf62ce7e6a742de68 not found: ID does not exist" Jul 23 18:52:50.624874 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.624844 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8475c8c8-svnhp"] Jul 23 18:52:50.639959 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:50.639935 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c8475c8c8-svnhp"] Jul 23 18:52:51.921328 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:51.921294 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d8759f-b8dc-461f-8e67-df6243808a90" path="/var/lib/kubelet/pods/99d8759f-b8dc-461f-8e67-df6243808a90/volumes" Jul 23 18:52:55.552660 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:55.552616 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:55.705490 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:52:55.705454 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:22.300064 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:22.299981 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:22.399558 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:22.399522 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:27.423337 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.423304 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76f54b79fb-b5hj6"] Jul 23 18:53:27.423728 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.423570 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d8759f-b8dc-461f-8e67-df6243808a90" containerName="console" Jul 23 18:53:27.423728 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.423580 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d8759f-b8dc-461f-8e67-df6243808a90" containerName="console" Jul 23 18:53:27.423728 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.423639 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d8759f-b8dc-461f-8e67-df6243808a90" containerName="console" Jul 23 18:53:27.428467 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.428442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.431352 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.431331 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 18:53:27.431456 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.431437 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-nsvk8\"" Jul 23 18:53:27.431882 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.431866 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 18:53:27.432036 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.432008 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 18:53:27.432466 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.432448 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 18:53:27.432619 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.432598 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 18:53:27.439738 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.439719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 18:53:27.443234 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.443215 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76f54b79fb-b5hj6"] Jul 23 18:53:27.508387 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508356 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-telemeter-client-tls\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-federate-client-tls\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkr7f\" (UniqueName: \"kubernetes.io/projected/a622d335-309c-4f22-8cd1-ca53cef7972d-kube-api-access-nkr7f\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-metrics-client-ca\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508680 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508564 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-serving-certs-ca-bundle\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.508680 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.508589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-secret-telemeter-client\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.609913 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.609855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610106 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.609929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-metrics-client-ca\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610106 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.609990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-serving-certs-ca-bundle\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610106 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.610027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-secret-telemeter-client\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610106 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.610073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-telemeter-client-tls\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610302 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.610215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-federate-client-tls\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610302 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.610269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkr7f\" (UniqueName: \"kubernetes.io/projected/a622d335-309c-4f22-8cd1-ca53cef7972d-kube-api-access-nkr7f\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.610404 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.610316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.611107 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.611056 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-serving-certs-ca-bundle\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.611290 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.611267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.611546 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.611529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a622d335-309c-4f22-8cd1-ca53cef7972d-metrics-client-ca\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.612882 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.612860 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-federate-client-tls\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.613099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.613078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.613213 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.613193 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-secret-telemeter-client\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.613287 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.613265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a622d335-309c-4f22-8cd1-ca53cef7972d-telemeter-client-tls\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.625123 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.625099 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkr7f\" (UniqueName: \"kubernetes.io/projected/a622d335-309c-4f22-8cd1-ca53cef7972d-kube-api-access-nkr7f\") pod \"telemeter-client-76f54b79fb-b5hj6\" (UID: \"a622d335-309c-4f22-8cd1-ca53cef7972d\") " pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.738367 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.738286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" Jul 23 18:53:27.899711 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:27.899688 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76f54b79fb-b5hj6"] Jul 23 18:53:27.901529 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:53:27.901506 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda622d335_309c_4f22_8cd1_ca53cef7972d.slice/crio-d150db29331a0cc1a2b1933427d9bb39e156a894264f5e434e0b48fac85282bf WatchSource:0}: Error finding container d150db29331a0cc1a2b1933427d9bb39e156a894264f5e434e0b48fac85282bf: Status 404 returned error can't find the container with id d150db29331a0cc1a2b1933427d9bb39e156a894264f5e434e0b48fac85282bf Jul 23 18:53:28.713777 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:28.713742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" event={"ID":"a622d335-309c-4f22-8cd1-ca53cef7972d","Type":"ContainerStarted","Data":"d150db29331a0cc1a2b1933427d9bb39e156a894264f5e434e0b48fac85282bf"} Jul 23 18:53:30.722283 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:30.722248 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" event={"ID":"a622d335-309c-4f22-8cd1-ca53cef7972d","Type":"ContainerStarted","Data":"ee58d62f3a74d089b2f21f114f2de73914fa3f53d3f995cfa3cbf4f801ae26e2"} Jul 23 18:53:31.727364 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:31.727327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" event={"ID":"a622d335-309c-4f22-8cd1-ca53cef7972d","Type":"ContainerStarted","Data":"ca7a71d1c9caa7a62b7ef72e33fd1c01a2935c3b9dd56da22f29415d742c98ec"} Jul 23 18:53:31.727771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:31.727388 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" event={"ID":"a622d335-309c-4f22-8cd1-ca53cef7972d","Type":"ContainerStarted","Data":"2d9a6f9b9f3c1839975758def85972efc1b790d50e1f576d13a75b16734cb71f"} Jul 23 18:53:32.507310 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.507256 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76f54b79fb-b5hj6" podStartSLOduration=2.734169048 podStartE2EDuration="5.507237381s" totalImagesPullingTime="2.773068333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:53:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:53:31.762111089 +0000 UTC m=+164.343255319" watchObservedRunningTime="2026-07-23 18:53:32.507237381 +0000 UTC m=+165.088381602" Jul 23 18:53:32.507552 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.507539 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-594f47984c-n6fn4"] Jul 23 18:53:32.510766 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.510749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.555840 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.555791 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-594f47984c-n6fn4"] Jul 23 18:53:32.611712 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.611681 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:32.652846 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.652795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-oauth-config\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.653041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.652882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-serving-cert\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.653041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.652950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-oauth-serving-cert\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.653041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.652990 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-trusted-ca-bundle\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.653041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.653028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-service-ca\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.653041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.653045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sw2p\" (UniqueName: \"kubernetes.io/projected/2178eb55-672e-457a-b138-eec4fdb03fcf-kube-api-access-7sw2p\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.653223 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.653180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-console-config\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.753717 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-oauth-config\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.753717 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-serving-cert\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.754268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-oauth-serving-cert\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.754268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-trusted-ca-bundle\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.754268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-service-ca\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.754268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sw2p\" (UniqueName: \"kubernetes.io/projected/2178eb55-672e-457a-b138-eec4fdb03fcf-kube-api-access-7sw2p\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.754268 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.753981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-console-config\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.759899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.755171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-oauth-serving-cert\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.759899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.755468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-service-ca\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.759899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.755670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-console-config\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.759899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.755700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-trusted-ca-bundle\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.759899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.756997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-oauth-config\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.759899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.757424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-serving-cert\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.766515 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.766490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sw2p\" (UniqueName: \"kubernetes.io/projected/2178eb55-672e-457a-b138-eec4fdb03fcf-kube-api-access-7sw2p\") pod \"console-594f47984c-n6fn4\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.774108 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.774085 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:32.820291 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.820247 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:32.967277 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:32.966345 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-594f47984c-n6fn4"] Jul 23 18:53:32.968727 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:53:32.968695 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2178eb55_672e_457a_b138_eec4fdb03fcf.slice/crio-caea5209dc8d3bc05690b22ec323c71efe98d302be2d38be05d18a45a78ae735 WatchSource:0}: Error finding container caea5209dc8d3bc05690b22ec323c71efe98d302be2d38be05d18a45a78ae735: Status 404 returned error can't find the container with id caea5209dc8d3bc05690b22ec323c71efe98d302be2d38be05d18a45a78ae735 Jul 23 18:53:33.737153 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:33.737114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f47984c-n6fn4" event={"ID":"2178eb55-672e-457a-b138-eec4fdb03fcf","Type":"ContainerStarted","Data":"fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6"} Jul 23 18:53:33.737328 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:33.737165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f47984c-n6fn4" event={"ID":"2178eb55-672e-457a-b138-eec4fdb03fcf","Type":"ContainerStarted","Data":"caea5209dc8d3bc05690b22ec323c71efe98d302be2d38be05d18a45a78ae735"} Jul 23 18:53:33.772994 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:33.772942 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-594f47984c-n6fn4" podStartSLOduration=1.772924618 podStartE2EDuration="1.772924618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:53:33.772525457 +0000 UTC m=+166.353669690" watchObservedRunningTime="2026-07-23 18:53:33.772924618 +0000 UTC m=+166.354068839" Jul 23 18:53:42.821053 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:42.820951 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:42.821053 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:42.821017 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:42.825734 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:42.825711 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:43.775955 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:43.775925 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:53:43.830939 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:43.830909 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb8c85cc4-4pdv6"] Jul 23 18:53:44.032837 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:44.032750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:44.163276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:44.163226 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:44.232493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:53:44.232464 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:08.851922 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:08.851879 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cb8c85cc4-4pdv6" podUID="759e47d5-b00f-49cb-908a-be9005620df6" containerName="console" containerID="cri-o://188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009" gracePeriod=15 Jul 23 18:54:09.088658 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.088631 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:54:09.147320 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147262 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-serving-cert\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147330 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-trusted-ca-bundle\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147350 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-oauth-serving-cert\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147373 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-oauth-config\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147390 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-service-ca\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147411 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-console-config\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147668 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147437 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsppr\" (UniqueName: \"kubernetes.io/projected/759e47d5-b00f-49cb-908a-be9005620df6-kube-api-access-nsppr\") pod \"759e47d5-b00f-49cb-908a-be9005620df6\" (UID: \"759e47d5-b00f-49cb-908a-be9005620df6\") " Jul 23 18:54:09.147943 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147875 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:54:09.147943 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.147906 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-service-ca" (OuterVolumeSpecName: "service-ca") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:54:09.148259 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.148002 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:54:09.148259 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.148162 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-console-config" (OuterVolumeSpecName: "console-config") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:54:09.149465 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.149442 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:54:09.149651 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.149632 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/759e47d5-b00f-49cb-908a-be9005620df6-kube-api-access-nsppr" (OuterVolumeSpecName: "kube-api-access-nsppr") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "kube-api-access-nsppr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:54:09.149651 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.149634 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "759e47d5-b00f-49cb-908a-be9005620df6" (UID: "759e47d5-b00f-49cb-908a-be9005620df6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:54:09.247936 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247917 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsppr\" (UniqueName: \"kubernetes.io/projected/759e47d5-b00f-49cb-908a-be9005620df6-kube-api-access-nsppr\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.247936 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247936 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.248059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247946 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-trusted-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.248059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247955 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-oauth-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.248059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247964 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/759e47d5-b00f-49cb-908a-be9005620df6-console-oauth-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.248059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247972 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.248059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.247980 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/759e47d5-b00f-49cb-908a-be9005620df6-console-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:54:09.866353 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.866314 2578 generic.go:356] "Generic (PLEG): container finished" podID="759e47d5-b00f-49cb-908a-be9005620df6" containerID="188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009" exitCode=0 Jul 23 18:54:09.866792 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.866383 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb8c85cc4-4pdv6" Jul 23 18:54:09.866792 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.866380 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb8c85cc4-4pdv6" event={"ID":"759e47d5-b00f-49cb-908a-be9005620df6","Type":"ContainerDied","Data":"188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009"} Jul 23 18:54:09.866792 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.866524 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb8c85cc4-4pdv6" event={"ID":"759e47d5-b00f-49cb-908a-be9005620df6","Type":"ContainerDied","Data":"6ddcd2548448e73a5f6008af02f578309ee3e7456d8f507b07b4e0b23a8ef08c"} Jul 23 18:54:09.866792 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.866552 2578 scope.go:117] "RemoveContainer" containerID="188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009" Jul 23 18:54:09.874855 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.874809 2578 scope.go:117] "RemoveContainer" containerID="188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009" Jul 23 18:54:09.875102 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:54:09.875083 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009\": container with ID starting with 188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009 not found: ID does not exist" containerID="188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009" Jul 23 18:54:09.875165 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.875111 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009"} err="failed to get container status \"188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009\": rpc error: code = NotFound desc = could not find container \"188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009\": container with ID starting with 188a626980e6104c7a72bc344f963618e4c69b6a9291b2324e6bd1676c3d3009 not found: ID does not exist" Jul 23 18:54:09.891755 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.891730 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb8c85cc4-4pdv6"] Jul 23 18:54:09.898238 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.898218 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cb8c85cc4-4pdv6"] Jul 23 18:54:09.923167 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:09.923141 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="759e47d5-b00f-49cb-908a-be9005620df6" path="/var/lib/kubelet/pods/759e47d5-b00f-49cb-908a-be9005620df6/volumes" Jul 23 18:54:25.624902 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:25.624861 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:25.781380 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:25.781346 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:29.214524 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:29.214491 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:37.782973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:37.782937 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:37.884525 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:37.884483 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:39.652631 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.652601 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77dd4b66db-5f48g"] Jul 23 18:54:39.653032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.652896 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="759e47d5-b00f-49cb-908a-be9005620df6" containerName="console" Jul 23 18:54:39.653032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.652907 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="759e47d5-b00f-49cb-908a-be9005620df6" containerName="console" Jul 23 18:54:39.653032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.652961 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="759e47d5-b00f-49cb-908a-be9005620df6" containerName="console" Jul 23 18:54:39.655695 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.655680 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.669771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.669749 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dd4b66db-5f48g"] Jul 23 18:54:39.705443 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.705416 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:39.765037 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-serving-cert\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.765201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf2bb\" (UniqueName: \"kubernetes.io/projected/22daede8-67cb-4ef7-9ffb-79854a109eb7-kube-api-access-wf2bb\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.765201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-trusted-ca-bundle\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.765201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-config\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.765201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-service-ca\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.765201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-oauth-serving-cert\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.765398 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.765243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-oauth-config\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.780651 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.780623 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:39.866471 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf2bb\" (UniqueName: \"kubernetes.io/projected/22daede8-67cb-4ef7-9ffb-79854a109eb7-kube-api-access-wf2bb\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.866599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-trusted-ca-bundle\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.866599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-config\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.866599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-service-ca\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.866599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-oauth-serving-cert\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.866794 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-oauth-config\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.866794 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.866671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-serving-cert\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.873006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.867919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-oauth-serving-cert\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.873006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.867943 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-service-ca\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.873006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.868097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-config\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.873006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.869696 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-serving-cert\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.873006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.870155 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-oauth-config\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.873553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.873532 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-trusted-ca-bundle\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.876031 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.876013 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf2bb\" (UniqueName: \"kubernetes.io/projected/22daede8-67cb-4ef7-9ffb-79854a109eb7-kube-api-access-wf2bb\") pod \"console-77dd4b66db-5f48g\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:39.965090 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:39.965019 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:40.100637 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:40.100606 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dd4b66db-5f48g"] Jul 23 18:54:40.104129 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:54:40.104096 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22daede8_67cb_4ef7_9ffb_79854a109eb7.slice/crio-f3a27887175681c8acca0011f0ed21e93da7cb81f54a0714427af23f6083f27c WatchSource:0}: Error finding container f3a27887175681c8acca0011f0ed21e93da7cb81f54a0714427af23f6083f27c: Status 404 returned error can't find the container with id f3a27887175681c8acca0011f0ed21e93da7cb81f54a0714427af23f6083f27c Jul 23 18:54:40.982392 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:40.982355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dd4b66db-5f48g" event={"ID":"22daede8-67cb-4ef7-9ffb-79854a109eb7","Type":"ContainerStarted","Data":"a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca"} Jul 23 18:54:40.982771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:40.982409 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dd4b66db-5f48g" event={"ID":"22daede8-67cb-4ef7-9ffb-79854a109eb7","Type":"ContainerStarted","Data":"f3a27887175681c8acca0011f0ed21e93da7cb81f54a0714427af23f6083f27c"} Jul 23 18:54:41.008351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:41.008303 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77dd4b66db-5f48g" podStartSLOduration=2.008287612 podStartE2EDuration="2.008287612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:54:41.006888694 +0000 UTC m=+233.588032914" watchObservedRunningTime="2026-07-23 18:54:41.008287612 +0000 UTC m=+233.589431914" Jul 23 18:54:49.796329 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:49.796288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:49.923078 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:49.923050 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:49.965351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:49.965326 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:49.965514 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:49.965376 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:49.969613 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:49.969594 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:50.016334 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:50.016307 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:54:50.116320 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:50.116285 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-594f47984c-n6fn4"] Jul 23 18:54:50.343033 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:50.342997 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:50.499996 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:50.499920 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:57.974079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:57.974044 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:54:58.074373 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:54:58.074336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:02.343599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:02.343558 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:02.544052 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:02.544017 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:15.136689 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.136608 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-594f47984c-n6fn4" podUID="2178eb55-672e-457a-b138-eec4fdb03fcf" containerName="console" containerID="cri-o://fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6" gracePeriod=15 Jul 23 18:55:15.381911 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.381888 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:55:15.443074 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443013 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-service-ca\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443074 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443069 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-serving-cert\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443258 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443093 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-oauth-serving-cert\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443258 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443126 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-oauth-config\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443258 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443145 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-console-config\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443258 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443172 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-trusted-ca-bundle\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443454 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443293 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sw2p\" (UniqueName: \"kubernetes.io/projected/2178eb55-672e-457a-b138-eec4fdb03fcf-kube-api-access-7sw2p\") pod \"2178eb55-672e-457a-b138-eec4fdb03fcf\" (UID: \"2178eb55-672e-457a-b138-eec4fdb03fcf\") " Jul 23 18:55:15.443509 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443480 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-service-ca" (OuterVolumeSpecName: "service-ca") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:55:15.443566 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443513 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:55:15.443617 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443579 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-console-config" (OuterVolumeSpecName: "console-config") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:55:15.443840 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.443794 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:55:15.445431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.445399 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2178eb55-672e-457a-b138-eec4fdb03fcf-kube-api-access-7sw2p" (OuterVolumeSpecName: "kube-api-access-7sw2p") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "kube-api-access-7sw2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:55:15.445431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.445420 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:55:15.445545 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.445435 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2178eb55-672e-457a-b138-eec4fdb03fcf" (UID: "2178eb55-672e-457a-b138-eec4fdb03fcf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:55:15.543883 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543860 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-oauth-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:15.543883 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543881 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-console-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:15.544006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543890 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-trusted-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:15.544006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543898 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sw2p\" (UniqueName: \"kubernetes.io/projected/2178eb55-672e-457a-b138-eec4fdb03fcf-kube-api-access-7sw2p\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:15.544006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543907 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:15.544006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543915 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2178eb55-672e-457a-b138-eec4fdb03fcf-console-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:15.544006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:15.543924 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2178eb55-672e-457a-b138-eec4fdb03fcf-oauth-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:16.104220 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.104183 2578 generic.go:356] "Generic (PLEG): container finished" podID="2178eb55-672e-457a-b138-eec4fdb03fcf" containerID="fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6" exitCode=0 Jul 23 18:55:16.104382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.104235 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f47984c-n6fn4" Jul 23 18:55:16.104382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.104268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f47984c-n6fn4" event={"ID":"2178eb55-672e-457a-b138-eec4fdb03fcf","Type":"ContainerDied","Data":"fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6"} Jul 23 18:55:16.104382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.104313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f47984c-n6fn4" event={"ID":"2178eb55-672e-457a-b138-eec4fdb03fcf","Type":"ContainerDied","Data":"caea5209dc8d3bc05690b22ec323c71efe98d302be2d38be05d18a45a78ae735"} Jul 23 18:55:16.104382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.104332 2578 scope.go:117] "RemoveContainer" containerID="fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6" Jul 23 18:55:16.113274 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.113260 2578 scope.go:117] "RemoveContainer" containerID="fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6" Jul 23 18:55:16.113812 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:55:16.113762 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6\": container with ID starting with fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6 not found: ID does not exist" containerID="fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6" Jul 23 18:55:16.113994 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.113972 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6"} err="failed to get container status \"fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6\": rpc error: code = NotFound desc = could not find container \"fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6\": container with ID starting with fd122840e4c377efcc3424094e007edd3e25477a3bd393ae5095b3d6e30811b6 not found: ID does not exist" Jul 23 18:55:16.121907 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.121884 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-594f47984c-n6fn4"] Jul 23 18:55:16.125723 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:16.125704 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-594f47984c-n6fn4"] Jul 23 18:55:17.921309 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:17.921276 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2178eb55-672e-457a-b138-eec4fdb03fcf" path="/var/lib/kubelet/pods/2178eb55-672e-457a-b138-eec4fdb03fcf/volumes" Jul 23 18:55:21.508068 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:21.508035 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:21.615830 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:21.615798 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:22.167767 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.167735 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9"] Jul 23 18:55:22.168041 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.168029 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2178eb55-672e-457a-b138-eec4fdb03fcf" containerName="console" Jul 23 18:55:22.168087 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.168043 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2178eb55-672e-457a-b138-eec4fdb03fcf" containerName="console" Jul 23 18:55:22.168120 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.168104 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2178eb55-672e-457a-b138-eec4fdb03fcf" containerName="console" Jul 23 18:55:22.172529 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.172514 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.175985 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.175965 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 18:55:22.176112 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.176036 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-44dtq\"" Jul 23 18:55:22.176893 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.176879 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 18:55:22.181014 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.180994 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9"] Jul 23 18:55:22.290687 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.290650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.290874 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.290701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.290874 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.290760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddtw2\" (UniqueName: \"kubernetes.io/projected/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-kube-api-access-ddtw2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.392088 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.392057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.392245 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.392109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddtw2\" (UniqueName: \"kubernetes.io/projected/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-kube-api-access-ddtw2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.392245 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.392148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.392417 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.392399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.392452 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.392428 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.402331 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.402302 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddtw2\" (UniqueName: \"kubernetes.io/projected/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-kube-api-access-ddtw2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.482383 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.482308 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:22.617267 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:22.617236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9"] Jul 23 18:55:22.620295 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:55:22.620267 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f3f707_05e7_436d_91ad_1d5ff5899ea8.slice/crio-2e112c96f51d617fb1fdfdf84e9ff7ee0d76e3f362598a47dd0a51f869497a9c WatchSource:0}: Error finding container 2e112c96f51d617fb1fdfdf84e9ff7ee0d76e3f362598a47dd0a51f869497a9c: Status 404 returned error can't find the container with id 2e112c96f51d617fb1fdfdf84e9ff7ee0d76e3f362598a47dd0a51f869497a9c Jul 23 18:55:23.132626 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:23.132591 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" event={"ID":"f4f3f707-05e7-436d-91ad-1d5ff5899ea8","Type":"ContainerStarted","Data":"2e112c96f51d617fb1fdfdf84e9ff7ee0d76e3f362598a47dd0a51f869497a9c"} Jul 23 18:55:30.167075 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:30.167029 2578 generic.go:356] "Generic (PLEG): container finished" podID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerID="8d7fa845bcbecbb692037b7e642df6437aa65b9c42a863aea36c1cc42c80b69e" exitCode=0 Jul 23 18:55:30.167461 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:30.167100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" event={"ID":"f4f3f707-05e7-436d-91ad-1d5ff5899ea8","Type":"ContainerDied","Data":"8d7fa845bcbecbb692037b7e642df6437aa65b9c42a863aea36c1cc42c80b69e"} Jul 23 18:55:30.648699 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:30.648664 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:30.752604 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:30.752574 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:33.180087 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:33.180054 2578 generic.go:356] "Generic (PLEG): container finished" podID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerID="268a5f55b2e24b4ea518b53766f58c9bd0e88cd9169e2aed7071447fb96a16e5" exitCode=0 Jul 23 18:55:33.180464 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:33.180119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" event={"ID":"f4f3f707-05e7-436d-91ad-1d5ff5899ea8","Type":"ContainerDied","Data":"268a5f55b2e24b4ea518b53766f58c9bd0e88cd9169e2aed7071447fb96a16e5"} Jul 23 18:55:40.211648 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:40.211620 2578 generic.go:356] "Generic (PLEG): container finished" podID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerID="a2c1e1b9001674612f1421f10f6d91efb3607ecbc9acbe30a4fa4f6e1ec2dc39" exitCode=0 Jul 23 18:55:40.211970 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:40.211695 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" event={"ID":"f4f3f707-05e7-436d-91ad-1d5ff5899ea8","Type":"ContainerDied","Data":"a2c1e1b9001674612f1421f10f6d91efb3607ecbc9acbe30a4fa4f6e1ec2dc39"} Jul 23 18:55:41.346326 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.346302 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:41.455489 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.455454 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-util\") pod \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " Jul 23 18:55:41.455489 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.455497 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddtw2\" (UniqueName: \"kubernetes.io/projected/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-kube-api-access-ddtw2\") pod \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " Jul 23 18:55:41.455699 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.455545 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-bundle\") pod \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\" (UID: \"f4f3f707-05e7-436d-91ad-1d5ff5899ea8\") " Jul 23 18:55:41.456231 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.456205 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-bundle" (OuterVolumeSpecName: "bundle") pod "f4f3f707-05e7-436d-91ad-1d5ff5899ea8" (UID: "f4f3f707-05e7-436d-91ad-1d5ff5899ea8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:55:41.457963 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.457935 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-kube-api-access-ddtw2" (OuterVolumeSpecName: "kube-api-access-ddtw2") pod "f4f3f707-05e7-436d-91ad-1d5ff5899ea8" (UID: "f4f3f707-05e7-436d-91ad-1d5ff5899ea8"). InnerVolumeSpecName "kube-api-access-ddtw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:55:41.460166 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.460142 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-util" (OuterVolumeSpecName: "util") pod "f4f3f707-05e7-436d-91ad-1d5ff5899ea8" (UID: "f4f3f707-05e7-436d-91ad-1d5ff5899ea8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:55:41.556505 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.556422 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:41.556505 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.556451 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddtw2\" (UniqueName: \"kubernetes.io/projected/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-kube-api-access-ddtw2\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:41.556505 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:41.556460 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f3f707-05e7-436d-91ad-1d5ff5899ea8-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:55:42.220192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:42.220099 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" event={"ID":"f4f3f707-05e7-436d-91ad-1d5ff5899ea8","Type":"ContainerDied","Data":"2e112c96f51d617fb1fdfdf84e9ff7ee0d76e3f362598a47dd0a51f869497a9c"} Jul 23 18:55:42.220192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:42.220151 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e112c96f51d617fb1fdfdf84e9ff7ee0d76e3f362598a47dd0a51f869497a9c" Jul 23 18:55:42.220192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:42.220160 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406chh9" Jul 23 18:55:43.788205 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788175 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf"] Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788436 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="pull" Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788448 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="pull" Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788459 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="util" Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788464 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="util" Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788479 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="extract" Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788484 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="extract" Jul 23 18:55:43.788576 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.788528 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f3f707-05e7-436d-91ad-1d5ff5899ea8" containerName="extract" Jul 23 18:55:43.819996 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.819967 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf"] Jul 23 18:55:43.820143 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.820055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:43.825127 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.825097 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 18:55:43.825253 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.825220 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 23 18:55:43.826150 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.826122 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 18:55:43.826269 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.826156 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 18:55:43.828562 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.828543 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:43.867919 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.867899 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7"] Jul 23 18:55:43.889516 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.889495 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7"] Jul 23 18:55:43.889630 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.889608 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:43.894057 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.894038 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 23 18:55:43.894421 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.894404 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 23 18:55:43.894494 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.894405 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 23 18:55:43.894856 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.894840 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 23 18:55:43.934909 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.934869 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:43.978114 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-ca\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:43.978225 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8p9\" (UniqueName: \"kubernetes.io/projected/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-kube-api-access-xq8p9\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:43.978225 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:43.978225 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:43.978329 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e86f759d-52f1-4467-b872-b52be9209d1e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc4756b88-hjfcf\" (UID: \"e86f759d-52f1-4467-b872-b52be9209d1e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:43.978329 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-hub\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:43.978390 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2q79\" (UniqueName: \"kubernetes.io/projected/e86f759d-52f1-4467-b872-b52be9209d1e-kube-api-access-m2q79\") pod \"managed-serviceaccount-addon-agent-6bc4756b88-hjfcf\" (UID: \"e86f759d-52f1-4467-b872-b52be9209d1e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:43.978390 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:43.978354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.078711 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8p9\" (UniqueName: \"kubernetes.io/projected/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-kube-api-access-xq8p9\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.078711 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.078973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.078973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e86f759d-52f1-4467-b872-b52be9209d1e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc4756b88-hjfcf\" (UID: \"e86f759d-52f1-4467-b872-b52be9209d1e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:44.078973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-hub\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.078973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2q79\" (UniqueName: \"kubernetes.io/projected/e86f759d-52f1-4467-b872-b52be9209d1e-kube-api-access-m2q79\") pod \"managed-serviceaccount-addon-agent-6bc4756b88-hjfcf\" (UID: \"e86f759d-52f1-4467-b872-b52be9209d1e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:44.078973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.079219 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.078971 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-ca\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.079975 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.079951 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.081896 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.081868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-ca\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.081974 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.081893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.081974 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.081944 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-hub\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.082137 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.082122 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e86f759d-52f1-4467-b872-b52be9209d1e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc4756b88-hjfcf\" (UID: \"e86f759d-52f1-4467-b872-b52be9209d1e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:44.082233 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.082214 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.089517 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.089492 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2q79\" (UniqueName: \"kubernetes.io/projected/e86f759d-52f1-4467-b872-b52be9209d1e-kube-api-access-m2q79\") pod \"managed-serviceaccount-addon-agent-6bc4756b88-hjfcf\" (UID: \"e86f759d-52f1-4467-b872-b52be9209d1e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:44.089683 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.089665 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8p9\" (UniqueName: \"kubernetes.io/projected/49a9a067-5b77-4f9c-8a9a-e3d49e903f0c-kube-api-access-xq8p9\") pod \"cluster-proxy-proxy-agent-75fc9856f7-df9x7\" (UID: \"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.140606 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.140581 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" Jul 23 18:55:44.199555 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.199527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" Jul 23 18:55:44.293849 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.293803 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf"] Jul 23 18:55:44.296705 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:55:44.296677 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode86f759d_52f1_4467_b872_b52be9209d1e.slice/crio-47ea34ce133221a97416edc12fe1939bf398cbc204bd460e8dedcf6078bd185d WatchSource:0}: Error finding container 47ea34ce133221a97416edc12fe1939bf398cbc204bd460e8dedcf6078bd185d: Status 404 returned error can't find the container with id 47ea34ce133221a97416edc12fe1939bf398cbc204bd460e8dedcf6078bd185d Jul 23 18:55:44.359230 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:44.359206 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7"] Jul 23 18:55:44.361332 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:55:44.361310 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49a9a067_5b77_4f9c_8a9a_e3d49e903f0c.slice/crio-7dfae9def55c41f95447c7e45cbb9dbfc8674015cc5f278fae16b6238557a36b WatchSource:0}: Error finding container 7dfae9def55c41f95447c7e45cbb9dbfc8674015cc5f278fae16b6238557a36b: Status 404 returned error can't find the container with id 7dfae9def55c41f95447c7e45cbb9dbfc8674015cc5f278fae16b6238557a36b Jul 23 18:55:45.226346 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:45.226310 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:45.234241 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:45.234210 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" event={"ID":"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c","Type":"ContainerStarted","Data":"7dfae9def55c41f95447c7e45cbb9dbfc8674015cc5f278fae16b6238557a36b"} Jul 23 18:55:45.235414 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:45.235392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" event={"ID":"e86f759d-52f1-4467-b872-b52be9209d1e","Type":"ContainerStarted","Data":"47ea34ce133221a97416edc12fe1939bf398cbc204bd460e8dedcf6078bd185d"} Jul 23 18:55:48.241918 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:48.241877 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:48.252162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:48.252131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" event={"ID":"e86f759d-52f1-4467-b872-b52be9209d1e","Type":"ContainerStarted","Data":"5fd09fda8171f07b6857b92ded8a737400d361e696e0b32d2b889ccfe1c1c8ae"} Jul 23 18:55:48.288101 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:48.288040 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc4756b88-hjfcf" podStartSLOduration=2.296606601 podStartE2EDuration="5.288023316s" totalImagesPullingTime="2.991416715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:55:48.286982258 +0000 UTC m=+300.868126479" watchObservedRunningTime="2026-07-23 18:55:48.288023316 +0000 UTC m=+300.869167537" Jul 23 18:55:48.656428 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:48.656404 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 18:55:49.258475 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:49.258434 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" event={"ID":"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c","Type":"ContainerStarted","Data":"2eb44b06165a580edd6f9853564ab6af700b099c0311b945065cece2ea86023a"} Jul 23 18:55:49.258904 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:49.258493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" event={"ID":"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c","Type":"ContainerStarted","Data":"26d7ee69ab802644539e39e9c377b2748c8d2eee30bd9c5501d608eccd0318dc"} Jul 23 18:55:49.258904 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:49.258511 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" event={"ID":"49a9a067-5b77-4f9c-8a9a-e3d49e903f0c","Type":"ContainerStarted","Data":"44849066a1c6f7822eef68dc884a9bf6239de6f739b07e81ff7e8a01a64cec6d"} Jul 23 18:55:49.294203 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:49.294150 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75fc9856f7-df9x7" podStartSLOduration=1.978901462 podStartE2EDuration="6.294132489s" totalImagesPullingTime="4.315231027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:55:49.292866965 +0000 UTC m=+301.874011188" watchObservedRunningTime="2026-07-23 18:55:49.294132489 +0000 UTC m=+301.875276710" Jul 23 18:55:49.295032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:49.294990 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:51.958392 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:51.958358 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:52.491881 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:52.491842 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:57.472569 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:57.472540 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:55:57.597716 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:55:57.597651 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:13.650997 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:13.650938 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:13.797867 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:13.797812 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:13.985867 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:13.985766 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:14.092621 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.092590 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl"] Jul 23 18:56:14.112008 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.111977 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.112695 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.112675 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl"] Jul 23 18:56:14.115137 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.115112 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-44dtq\"" Jul 23 18:56:14.115930 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.115910 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 18:56:14.116408 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.116390 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 18:56:14.196110 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.196078 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcm5k\" (UniqueName: \"kubernetes.io/projected/fad92686-0e8a-43db-b03d-2a32e6a36c56-kube-api-access-bcm5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.196276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.196130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.196276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.196264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.296786 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.296702 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.296786 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.296763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.297017 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.296793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcm5k\" (UniqueName: \"kubernetes.io/projected/fad92686-0e8a-43db-b03d-2a32e6a36c56-kube-api-access-bcm5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.297169 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.297147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.297235 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.297176 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.310990 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.310961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcm5k\" (UniqueName: \"kubernetes.io/projected/fad92686-0e8a-43db-b03d-2a32e6a36c56-kube-api-access-bcm5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.422976 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.422953 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:14.577333 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:14.577302 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl"] Jul 23 18:56:14.580348 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:56:14.580319 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad92686_0e8a_43db_b03d_2a32e6a36c56.slice/crio-1f27092820978faaee8531b1fc591e4ccd576052f7b427d77553bc5c34271e3e WatchSource:0}: Error finding container 1f27092820978faaee8531b1fc591e4ccd576052f7b427d77553bc5c34271e3e: Status 404 returned error can't find the container with id 1f27092820978faaee8531b1fc591e4ccd576052f7b427d77553bc5c34271e3e Jul 23 18:56:15.354239 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:15.354211 2578 generic.go:356] "Generic (PLEG): container finished" podID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerID="4917cddcc4f745ff747b1cf9747b5e35316ed5044528d6cef88273abb8e934e5" exitCode=0 Jul 23 18:56:15.354571 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:15.354280 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" event={"ID":"fad92686-0e8a-43db-b03d-2a32e6a36c56","Type":"ContainerDied","Data":"4917cddcc4f745ff747b1cf9747b5e35316ed5044528d6cef88273abb8e934e5"} Jul 23 18:56:15.354571 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:15.354319 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" event={"ID":"fad92686-0e8a-43db-b03d-2a32e6a36c56","Type":"ContainerStarted","Data":"1f27092820978faaee8531b1fc591e4ccd576052f7b427d77553bc5c34271e3e"} Jul 23 18:56:15.355212 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:15.355196 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:56:15.838087 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:15.838053 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:16.248623 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:16.248507 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:18.367737 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:18.367700 2578 generic.go:356] "Generic (PLEG): container finished" podID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerID="c467a4cad822a88b2c2610f13386766bb6757994b20015420d498c6c79420e4b" exitCode=0 Jul 23 18:56:18.368166 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:18.367786 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" event={"ID":"fad92686-0e8a-43db-b03d-2a32e6a36c56","Type":"ContainerDied","Data":"c467a4cad822a88b2c2610f13386766bb6757994b20015420d498c6c79420e4b"} Jul 23 18:56:19.373632 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:19.373601 2578 generic.go:356] "Generic (PLEG): container finished" podID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerID="04c387fa47d195f60c788e4c1510640605a62db82c62d51047d2b010f337415a" exitCode=0 Jul 23 18:56:19.374032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:19.373677 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" event={"ID":"fad92686-0e8a-43db-b03d-2a32e6a36c56","Type":"ContainerDied","Data":"04c387fa47d195f60c788e4c1510640605a62db82c62d51047d2b010f337415a"} Jul 23 18:56:20.553808 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.553774 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:20.645919 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.645850 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-util\") pod \"fad92686-0e8a-43db-b03d-2a32e6a36c56\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " Jul 23 18:56:20.645919 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.645903 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-bundle\") pod \"fad92686-0e8a-43db-b03d-2a32e6a36c56\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " Jul 23 18:56:20.646087 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.645966 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcm5k\" (UniqueName: \"kubernetes.io/projected/fad92686-0e8a-43db-b03d-2a32e6a36c56-kube-api-access-bcm5k\") pod \"fad92686-0e8a-43db-b03d-2a32e6a36c56\" (UID: \"fad92686-0e8a-43db-b03d-2a32e6a36c56\") " Jul 23 18:56:20.646282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.646256 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-bundle" (OuterVolumeSpecName: "bundle") pod "fad92686-0e8a-43db-b03d-2a32e6a36c56" (UID: "fad92686-0e8a-43db-b03d-2a32e6a36c56"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:56:20.648083 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.648060 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad92686-0e8a-43db-b03d-2a32e6a36c56-kube-api-access-bcm5k" (OuterVolumeSpecName: "kube-api-access-bcm5k") pod "fad92686-0e8a-43db-b03d-2a32e6a36c56" (UID: "fad92686-0e8a-43db-b03d-2a32e6a36c56"). InnerVolumeSpecName "kube-api-access-bcm5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:56:20.652357 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.652334 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-util" (OuterVolumeSpecName: "util") pod "fad92686-0e8a-43db-b03d-2a32e6a36c56" (UID: "fad92686-0e8a-43db-b03d-2a32e6a36c56"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:56:20.747285 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.747257 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcm5k\" (UniqueName: \"kubernetes.io/projected/fad92686-0e8a-43db-b03d-2a32e6a36c56-kube-api-access-bcm5k\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:56:20.747285 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.747281 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:56:20.747285 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:20.747291 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad92686-0e8a-43db-b03d-2a32e6a36c56-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:56:21.382889 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:21.382864 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" Jul 23 18:56:21.382889 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:21.382880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z9pl" event={"ID":"fad92686-0e8a-43db-b03d-2a32e6a36c56","Type":"ContainerDied","Data":"1f27092820978faaee8531b1fc591e4ccd576052f7b427d77553bc5c34271e3e"} Jul 23 18:56:21.383093 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:21.382928 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f27092820978faaee8531b1fc591e4ccd576052f7b427d77553bc5c34271e3e" Jul 23 18:56:23.767282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:23.767244 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:24.291332 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:24.291301 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:24.389227 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:24.389199 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:24.492652 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:24.492621 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:25.593704 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.593674 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj"] Jul 23 18:56:25.594184 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594106 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="util" Jul 23 18:56:25.594184 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594125 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="util" Jul 23 18:56:25.594184 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594140 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="extract" Jul 23 18:56:25.594184 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594148 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="extract" Jul 23 18:56:25.594184 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594174 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="pull" Jul 23 18:56:25.594184 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594182 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="pull" Jul 23 18:56:25.594464 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.594254 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad92686-0e8a-43db-b03d-2a32e6a36c56" containerName="extract" Jul 23 18:56:25.596621 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.596603 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.600039 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.600017 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:56:25.600388 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.600373 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 18:56:25.600460 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.600374 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wwtf5\"" Jul 23 18:56:25.608472 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.608450 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj"] Jul 23 18:56:25.683427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.683399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30d98f8f-f05f-4ca5-b860-0d0c9ab8a250-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qh2gj\" (UID: \"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.683539 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.683450 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nfgb\" (UniqueName: \"kubernetes.io/projected/30d98f8f-f05f-4ca5-b860-0d0c9ab8a250-kube-api-access-2nfgb\") pod \"openshift-lws-operator-fd8ccff4c-qh2gj\" (UID: \"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.784099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.784048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30d98f8f-f05f-4ca5-b860-0d0c9ab8a250-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qh2gj\" (UID: \"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.784248 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.784128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nfgb\" (UniqueName: \"kubernetes.io/projected/30d98f8f-f05f-4ca5-b860-0d0c9ab8a250-kube-api-access-2nfgb\") pod \"openshift-lws-operator-fd8ccff4c-qh2gj\" (UID: \"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.784451 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.784433 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30d98f8f-f05f-4ca5-b860-0d0c9ab8a250-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qh2gj\" (UID: \"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.793811 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.793780 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nfgb\" (UniqueName: \"kubernetes.io/projected/30d98f8f-f05f-4ca5-b860-0d0c9ab8a250-kube-api-access-2nfgb\") pod \"openshift-lws-operator-fd8ccff4c-qh2gj\" (UID: \"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:25.905501 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:25.905417 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" Jul 23 18:56:26.039076 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:26.039052 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj"] Jul 23 18:56:26.041148 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:56:26.041117 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30d98f8f_f05f_4ca5_b860_0d0c9ab8a250.slice/crio-87bdc6157c6a6a6d88dde9405723859b20714575409c7b0c247cda21bcaf6a95 WatchSource:0}: Error finding container 87bdc6157c6a6a6d88dde9405723859b20714575409c7b0c247cda21bcaf6a95: Status 404 returned error can't find the container with id 87bdc6157c6a6a6d88dde9405723859b20714575409c7b0c247cda21bcaf6a95 Jul 23 18:56:26.402890 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:26.402858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" event={"ID":"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250","Type":"ContainerStarted","Data":"87bdc6157c6a6a6d88dde9405723859b20714575409c7b0c247cda21bcaf6a95"} Jul 23 18:56:28.413254 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:28.413215 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" event={"ID":"30d98f8f-f05f-4ca5-b860-0d0c9ab8a250","Type":"ContainerStarted","Data":"71daa201e2c763c6b282516187608a7fa10284c7fc294b05d95991a2e1b50e1d"} Jul 23 18:56:28.435415 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:28.435357 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qh2gj" podStartSLOduration=1.852189113 podStartE2EDuration="3.435339328s" totalImagesPullingTime="1.583150215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:56:28.434077011 +0000 UTC m=+341.015221232" watchObservedRunningTime="2026-07-23 18:56:28.435339328 +0000 UTC m=+341.016483550" Jul 23 18:56:29.200265 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:29.200209 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:29.300068 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:29.300036 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:31.257893 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.257859 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8"] Jul 23 18:56:31.260104 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.260088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.263277 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.263258 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-44dtq\"" Jul 23 18:56:31.263658 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.263641 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 18:56:31.264196 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.264180 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 18:56:31.275585 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.275561 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8"] Jul 23 18:56:31.328151 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.328127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.328266 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.328188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.328266 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.328212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74prk\" (UniqueName: \"kubernetes.io/projected/f351d79f-b157-4aba-9408-8a641e583326-kube-api-access-74prk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.428973 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.428946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.429114 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.428980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74prk\" (UniqueName: \"kubernetes.io/projected/f351d79f-b157-4aba-9408-8a641e583326-kube-api-access-74prk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.429114 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.429012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.429413 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.429394 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.429451 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.429405 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.433308 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.433282 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:31.439801 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.439771 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74prk\" (UniqueName: \"kubernetes.io/projected/f351d79f-b157-4aba-9408-8a641e583326-kube-api-access-74prk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.569193 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.569118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:31.701036 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:31.700991 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8"] Jul 23 18:56:31.703842 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:56:31.703792 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf351d79f_b157_4aba_9408_8a641e583326.slice/crio-b9351a98c8d7c45b94289bde975a8423337fb3effd819747213526e9e2c22eb8 WatchSource:0}: Error finding container b9351a98c8d7c45b94289bde975a8423337fb3effd819747213526e9e2c22eb8: Status 404 returned error can't find the container with id b9351a98c8d7c45b94289bde975a8423337fb3effd819747213526e9e2c22eb8 Jul 23 18:56:32.429738 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:32.429704 2578 generic.go:356] "Generic (PLEG): container finished" podID="f351d79f-b157-4aba-9408-8a641e583326" containerID="ca0dadda494b2cce056697bc1c992883d8494d543331da39e259ecc98a7ec0fb" exitCode=0 Jul 23 18:56:32.430229 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:32.429782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" event={"ID":"f351d79f-b157-4aba-9408-8a641e583326","Type":"ContainerDied","Data":"ca0dadda494b2cce056697bc1c992883d8494d543331da39e259ecc98a7ec0fb"} Jul 23 18:56:32.430229 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:32.429816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" event={"ID":"f351d79f-b157-4aba-9408-8a641e583326","Type":"ContainerStarted","Data":"b9351a98c8d7c45b94289bde975a8423337fb3effd819747213526e9e2c22eb8"} Jul 23 18:56:33.436166 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:33.436082 2578 generic.go:356] "Generic (PLEG): container finished" podID="f351d79f-b157-4aba-9408-8a641e583326" containerID="1a5d43b5d3c9f2119cc8b35d1755590a26a673a53e0a1afa1e780cb1038ab034" exitCode=0 Jul 23 18:56:33.436594 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:33.436169 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" event={"ID":"f351d79f-b157-4aba-9408-8a641e583326","Type":"ContainerDied","Data":"1a5d43b5d3c9f2119cc8b35d1755590a26a673a53e0a1afa1e780cb1038ab034"} Jul 23 18:56:34.442519 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:34.442482 2578 generic.go:356] "Generic (PLEG): container finished" podID="f351d79f-b157-4aba-9408-8a641e583326" containerID="8b0032c82d2987efdec41643f767c48cb1d7d24a7a5a87a0576b61c8dc13fb2e" exitCode=0 Jul 23 18:56:34.442899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:34.442568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" event={"ID":"f351d79f-b157-4aba-9408-8a641e583326","Type":"ContainerDied","Data":"8b0032c82d2987efdec41643f767c48cb1d7d24a7a5a87a0576b61c8dc13fb2e"} Jul 23 18:56:35.579223 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.579203 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:35.665038 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.665011 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74prk\" (UniqueName: \"kubernetes.io/projected/f351d79f-b157-4aba-9408-8a641e583326-kube-api-access-74prk\") pod \"f351d79f-b157-4aba-9408-8a641e583326\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " Jul 23 18:56:35.665201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.665055 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-bundle\") pod \"f351d79f-b157-4aba-9408-8a641e583326\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " Jul 23 18:56:35.665201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.665097 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-util\") pod \"f351d79f-b157-4aba-9408-8a641e583326\" (UID: \"f351d79f-b157-4aba-9408-8a641e583326\") " Jul 23 18:56:35.665746 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.665724 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-bundle" (OuterVolumeSpecName: "bundle") pod "f351d79f-b157-4aba-9408-8a641e583326" (UID: "f351d79f-b157-4aba-9408-8a641e583326"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:56:35.667259 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.667232 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f351d79f-b157-4aba-9408-8a641e583326-kube-api-access-74prk" (OuterVolumeSpecName: "kube-api-access-74prk") pod "f351d79f-b157-4aba-9408-8a641e583326" (UID: "f351d79f-b157-4aba-9408-8a641e583326"). InnerVolumeSpecName "kube-api-access-74prk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:56:35.670874 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.670840 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-util" (OuterVolumeSpecName: "util") pod "f351d79f-b157-4aba-9408-8a641e583326" (UID: "f351d79f-b157-4aba-9408-8a641e583326"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:56:35.765816 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.765739 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74prk\" (UniqueName: \"kubernetes.io/projected/f351d79f-b157-4aba-9408-8a641e583326-kube-api-access-74prk\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:56:35.765816 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.765766 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:56:35.765816 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:35.765778 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f351d79f-b157-4aba-9408-8a641e583326-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:56:36.452866 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:36.452833 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" event={"ID":"f351d79f-b157-4aba-9408-8a641e583326","Type":"ContainerDied","Data":"b9351a98c8d7c45b94289bde975a8423337fb3effd819747213526e9e2c22eb8"} Jul 23 18:56:36.452866 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:36.452860 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afff7s8" Jul 23 18:56:36.453091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:36.452881 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9351a98c8d7c45b94289bde975a8423337fb3effd819747213526e9e2c22eb8" Jul 23 18:56:43.929382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:43.929303 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:44.104604 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:44.104571 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:47.191489 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.191457 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:47.342020 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.341953 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:47.625260 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.625229 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:47.787293 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.787266 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:47.893076 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.893004 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:47.922873 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.922843 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj"] Jul 23 18:56:47.923125 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923114 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="extract" Jul 23 18:56:47.923162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923126 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="extract" Jul 23 18:56:47.923162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923137 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="pull" Jul 23 18:56:47.923162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923142 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="pull" Jul 23 18:56:47.923162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923154 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="util" Jul 23 18:56:47.923162 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923159 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="util" Jul 23 18:56:47.923304 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.923199 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f351d79f-b157-4aba-9408-8a641e583326" containerName="extract" Jul 23 18:56:47.925993 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.925977 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:47.929602 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.929584 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 18:56:47.930045 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.930026 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 18:56:47.930190 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.930170 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 18:56:47.930340 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.930326 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 18:56:47.932493 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.932426 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zmrr8\"" Jul 23 18:56:47.956156 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:47.956125 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj"] Jul 23 18:56:48.070476 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.070442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0089e0-fc30-4c90-9a19-2978a6883436-manifests\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.070476 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.070477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0089e0-fc30-4c90-9a19-2978a6883436-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.070734 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.070502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0089e0-fc30-4c90-9a19-2978a6883436-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.070734 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.070520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0089e0-fc30-4c90-9a19-2978a6883436-tmp\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.070734 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.070625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zgl5\" (UniqueName: \"kubernetes.io/projected/5e0089e0-fc30-4c90-9a19-2978a6883436-kube-api-access-5zgl5\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.094003 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.093969 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:48.171282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.171191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0089e0-fc30-4c90-9a19-2978a6883436-manifests\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.171282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.171232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0089e0-fc30-4c90-9a19-2978a6883436-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.171282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.171250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0089e0-fc30-4c90-9a19-2978a6883436-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.171282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.171270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0089e0-fc30-4c90-9a19-2978a6883436-tmp\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.171578 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.171307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zgl5\" (UniqueName: \"kubernetes.io/projected/5e0089e0-fc30-4c90-9a19-2978a6883436-kube-api-access-5zgl5\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.171897 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.171851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0089e0-fc30-4c90-9a19-2978a6883436-manifests\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.177248 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.174799 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0089e0-fc30-4c90-9a19-2978a6883436-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.177248 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.176666 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0089e0-fc30-4c90-9a19-2978a6883436-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.177450 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.177429 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0089e0-fc30-4c90-9a19-2978a6883436-tmp\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.181591 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.181568 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zgl5\" (UniqueName: \"kubernetes.io/projected/5e0089e0-fc30-4c90-9a19-2978a6883436-kube-api-access-5zgl5\") pod \"opendatahub-operator-controller-manager-5c495ff4d9-lpxjj\" (UID: \"5e0089e0-fc30-4c90-9a19-2978a6883436\") " pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.195369 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.195337 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:48.235585 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.235557 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:48.292966 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.292931 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:48.402353 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.402330 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj"] Jul 23 18:56:48.404020 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:56:48.403985 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0089e0_fc30_4c90_9a19_2978a6883436.slice/crio-ce3b7eb7b1a92f76a138ba9af055802fdc93eb34e283cc9b0955ab76f5d5cdb4 WatchSource:0}: Error finding container ce3b7eb7b1a92f76a138ba9af055802fdc93eb34e283cc9b0955ab76f5d5cdb4: Status 404 returned error can't find the container with id ce3b7eb7b1a92f76a138ba9af055802fdc93eb34e283cc9b0955ab76f5d5cdb4 Jul 23 18:56:48.470343 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.470255 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:48.502737 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.502703 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" event={"ID":"5e0089e0-fc30-4c90-9a19-2978a6883436","Type":"ContainerStarted","Data":"ce3b7eb7b1a92f76a138ba9af055802fdc93eb34e283cc9b0955ab76f5d5cdb4"} Jul 23 18:56:48.568225 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.568190 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:48.671265 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:48.671213 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:56:52.525817 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:52.525775 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" event={"ID":"5e0089e0-fc30-4c90-9a19-2978a6883436","Type":"ContainerStarted","Data":"0a84233bb13a4a59a2f0c5281837abc1749c3526079edb21388463f8c3669750"} Jul 23 18:56:53.532158 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:53.532122 2578 generic.go:356] "Generic (PLEG): container finished" podID="5e0089e0-fc30-4c90-9a19-2978a6883436" containerID="0a84233bb13a4a59a2f0c5281837abc1749c3526079edb21388463f8c3669750" exitCode=0 Jul 23 18:56:53.532158 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:53.532161 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" event={"ID":"5e0089e0-fc30-4c90-9a19-2978a6883436","Type":"ContainerDied","Data":"0a84233bb13a4a59a2f0c5281837abc1749c3526079edb21388463f8c3669750"} Jul 23 18:56:54.539480 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:54.539441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" event={"ID":"5e0089e0-fc30-4c90-9a19-2978a6883436","Type":"ContainerStarted","Data":"725b23f7d67dc1a7f37a74f3d59e29e95cbec02cfa0a4cd170f33640322061b5"} Jul 23 18:56:54.539906 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:54.539611 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:56:54.570192 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:56:54.570146 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" podStartSLOduration=2.588875279 podStartE2EDuration="7.570129293s" totalImagesPullingTime="3.914534468s" totalInitContainerRuntime="1.066719546s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:56:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:56:54.569473552 +0000 UTC m=+367.150617772" watchObservedRunningTime="2026-07-23 18:56:54.570129293 +0000 UTC m=+367.151273513" Jul 23 18:57:05.372020 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:05.371960 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:05.472754 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:05.472713 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:05.546878 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:05.546855 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c495ff4d9-lpxjj" Jul 23 18:57:05.810223 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:05.810094 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:05.901401 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:05.901365 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:07.570447 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:07.570398 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:07.763384 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:07.763308 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:13.610445 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.610404 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:13.628305 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.628272 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8f784977d-7fbz7"] Jul 23 18:57:13.631636 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.631620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.638642 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.638618 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 18:57:13.638730 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.638647 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 18:57:13.638930 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.638916 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 18:57:13.639596 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.639579 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 18:57:13.650578 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.650554 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8f784977d-7fbz7"] Jul 23 18:57:13.678646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.678617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75dc15d6-0580-4360-97f0-a049f8851c2e-tls-certs\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.678784 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.678676 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj4d5\" (UniqueName: \"kubernetes.io/projected/75dc15d6-0580-4360-97f0-a049f8851c2e-kube-api-access-xj4d5\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.678784 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.678738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75dc15d6-0580-4360-97f0-a049f8851c2e-tmp\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.779632 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.779598 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nqrct"] Jul 23 18:57:13.779948 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.779924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75dc15d6-0580-4360-97f0-a049f8851c2e-tmp\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.780026 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.779983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75dc15d6-0580-4360-97f0-a049f8851c2e-tls-certs\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.780026 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.780023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj4d5\" (UniqueName: \"kubernetes.io/projected/75dc15d6-0580-4360-97f0-a049f8851c2e-kube-api-access-xj4d5\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.780147 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:57:13.780131 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 23 18:57:13.780206 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:57:13.780188 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc15d6-0580-4360-97f0-a049f8851c2e-tls-certs podName:75dc15d6-0580-4360-97f0-a049f8851c2e nodeName:}" failed. No retries permitted until 2026-07-23 18:57:14.280172468 +0000 UTC m=+386.861316667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/75dc15d6-0580-4360-97f0-a049f8851c2e-tls-certs") pod "kube-auth-proxy-8f784977d-7fbz7" (UID: "75dc15d6-0580-4360-97f0-a049f8851c2e") : secret "kube-auth-proxy-tls" not found Jul 23 18:57:13.782276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.782254 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75dc15d6-0580-4360-97f0-a049f8851c2e-tmp\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.782895 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.782881 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:13.787015 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.786995 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 18:57:13.787953 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.787936 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9pq5n\"" Jul 23 18:57:13.807712 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.807687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj4d5\" (UniqueName: \"kubernetes.io/projected/75dc15d6-0580-4360-97f0-a049f8851c2e-kube-api-access-xj4d5\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:13.808784 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.808763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nqrct"] Jul 23 18:57:13.837137 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.837113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:13.880394 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.880323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01252b47-6b6d-44ee-95c7-57d8e0fd1615-cert\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:13.880394 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.880393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4r7z\" (UniqueName: \"kubernetes.io/projected/01252b47-6b6d-44ee-95c7-57d8e0fd1615-kube-api-access-j4r7z\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:13.921217 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.921152 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:13.980954 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.980931 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4r7z\" (UniqueName: \"kubernetes.io/projected/01252b47-6b6d-44ee-95c7-57d8e0fd1615-kube-api-access-j4r7z\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:13.981048 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.980978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01252b47-6b6d-44ee-95c7-57d8e0fd1615-cert\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:13.981129 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:57:13.981117 2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 23 18:57:13.981189 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:57:13.981180 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01252b47-6b6d-44ee-95c7-57d8e0fd1615-cert podName:01252b47-6b6d-44ee-95c7-57d8e0fd1615 nodeName:}" failed. No retries permitted until 2026-07-23 18:57:14.481166592 +0000 UTC m=+387.062310790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/01252b47-6b6d-44ee-95c7-57d8e0fd1615-cert") pod "odh-model-controller-7998c67dc4-nqrct" (UID: "01252b47-6b6d-44ee-95c7-57d8e0fd1615") : secret "odh-model-controller-webhook-cert" not found Jul 23 18:57:13.994445 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:13.994417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4r7z\" (UniqueName: \"kubernetes.io/projected/01252b47-6b6d-44ee-95c7-57d8e0fd1615-kube-api-access-j4r7z\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:14.154433 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.154338 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:14.261677 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.261642 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:14.282944 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.282923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75dc15d6-0580-4360-97f0-a049f8851c2e-tls-certs\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:14.285356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.285328 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75dc15d6-0580-4360-97f0-a049f8851c2e-tls-certs\") pod \"kube-auth-proxy-8f784977d-7fbz7\" (UID: \"75dc15d6-0580-4360-97f0-a049f8851c2e\") " pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:14.484650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.484571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01252b47-6b6d-44ee-95c7-57d8e0fd1615-cert\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:14.487091 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.487062 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01252b47-6b6d-44ee-95c7-57d8e0fd1615-cert\") pod \"odh-model-controller-7998c67dc4-nqrct\" (UID: \"01252b47-6b6d-44ee-95c7-57d8e0fd1615\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:14.540812 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.540781 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" Jul 23 18:57:14.620012 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.619981 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:14.640972 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.640938 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-29h4l"] Jul 23 18:57:14.644944 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.644921 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:14.648916 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.648889 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 18:57:14.649102 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.649088 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7tzqw\"" Jul 23 18:57:14.658507 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.658482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-29h4l"] Jul 23 18:57:14.683299 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.683274 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8f784977d-7fbz7"] Jul 23 18:57:14.685794 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.685776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-tls-certs\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:14.685901 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.685803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjntt\" (UniqueName: \"kubernetes.io/projected/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-kube-api-access-mjntt\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:14.686365 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:14.686347 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75dc15d6_0580_4360_97f0_a049f8851c2e.slice/crio-6553e31cc1db08b52e0af0ad7c67dec0e2aa662fbd8ac2584295375e4d872a30 WatchSource:0}: Error finding container 6553e31cc1db08b52e0af0ad7c67dec0e2aa662fbd8ac2584295375e4d872a30: Status 404 returned error can't find the container with id 6553e31cc1db08b52e0af0ad7c67dec0e2aa662fbd8ac2584295375e4d872a30 Jul 23 18:57:14.698135 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.698117 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:14.786384 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.786346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-tls-certs\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:14.786522 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.786405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjntt\" (UniqueName: \"kubernetes.io/projected/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-kube-api-access-mjntt\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:14.786618 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:57:14.786595 2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 23 18:57:14.786692 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:57:14.786681 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-tls-certs podName:2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6 nodeName:}" failed. No retries permitted until 2026-07-23 18:57:15.286659719 +0000 UTC m=+387.867803916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-tls-certs") pod "model-serving-api-54d89bd5fc-29h4l" (UID: "2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6") : secret "model-serving-api-tls" not found Jul 23 18:57:14.802418 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.802396 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjntt\" (UniqueName: \"kubernetes.io/projected/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-kube-api-access-mjntt\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:14.835469 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:14.835444 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nqrct"] Jul 23 18:57:14.837232 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:14.837203 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01252b47_6b6d_44ee_95c7_57d8e0fd1615.slice/crio-4f0bdb3523f3dc220081538b0d9d0624d920ad488552aededdad0bf51332848a WatchSource:0}: Error finding container 4f0bdb3523f3dc220081538b0d9d0624d920ad488552aededdad0bf51332848a: Status 404 returned error can't find the container with id 4f0bdb3523f3dc220081538b0d9d0624d920ad488552aededdad0bf51332848a Jul 23 18:57:15.139805 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.139773 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:15.290140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.290114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-tls-certs\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:15.293169 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.293140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6-tls-certs\") pod \"model-serving-api-54d89bd5fc-29h4l\" (UID: \"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6\") " pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:15.378048 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.378018 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8"] Jul 23 18:57:15.382420 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.382404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.385509 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.385490 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 18:57:15.385665 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.385645 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:15.386195 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.386172 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 18:57:15.386317 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.386264 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 18:57:15.387035 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.387016 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 18:57:15.387130 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.387039 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-78nbt\"" Jul 23 18:57:15.391240 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-token\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.391240 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-cacerts\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.391359 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrbt5\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-kube-api-access-rrbt5\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.391359 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.391466 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.391466 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391454 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/df1b4903-7d42-45e3-8b9f-669671691d8f-local-certs\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.391561 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.391494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.404723 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.404705 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8"] Jul 23 18:57:15.486621 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.486591 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:15.492207 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/df1b4903-7d42-45e3-8b9f-669671691d8f-local-certs\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.492324 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.492324 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-token\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.492324 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-cacerts\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.492436 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrbt5\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-kube-api-access-rrbt5\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.492436 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.492436 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.492400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.493190 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.493163 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.495216 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.495171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/df1b4903-7d42-45e3-8b9f-669671691d8f-local-certs\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.495216 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.495192 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.495899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.495691 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.495899 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.495774 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-cacerts\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.501852 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.501812 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-token\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.503475 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.503446 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrbt5\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-kube-api-access-rrbt5\") pod \"istiod-openshift-gateway-7574459656-f2gf8\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.555045 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.555018 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:15.632624 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.632538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" event={"ID":"01252b47-6b6d-44ee-95c7-57d8e0fd1615","Type":"ContainerStarted","Data":"4f0bdb3523f3dc220081538b0d9d0624d920ad488552aededdad0bf51332848a"} Jul 23 18:57:15.634537 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.634479 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" event={"ID":"75dc15d6-0580-4360-97f0-a049f8851c2e","Type":"ContainerStarted","Data":"6553e31cc1db08b52e0af0ad7c67dec0e2aa662fbd8ac2584295375e4d872a30"} Jul 23 18:57:15.691923 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.691815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:15.724669 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.724635 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-29h4l"] Jul 23 18:57:15.727734 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:15.727701 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb4eafe_cd1c_41b5_9b63_41ba326d0cb6.slice/crio-726fef8f21585ebf3046ec97da38f513ee84e172927078fe8dd34e1a67bb5116 WatchSource:0}: Error finding container 726fef8f21585ebf3046ec97da38f513ee84e172927078fe8dd34e1a67bb5116: Status 404 returned error can't find the container with id 726fef8f21585ebf3046ec97da38f513ee84e172927078fe8dd34e1a67bb5116 Jul 23 18:57:15.865735 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:15.865704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8"] Jul 23 18:57:15.869749 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:15.869721 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1b4903_7d42_45e3_8b9f_669671691d8f.slice/crio-829daa69f13a6e8d65fe89102c99e213b3d704d3831ab7d9ea7e927d1c5d816a WatchSource:0}: Error finding container 829daa69f13a6e8d65fe89102c99e213b3d704d3831ab7d9ea7e927d1c5d816a: Status 404 returned error can't find the container with id 829daa69f13a6e8d65fe89102c99e213b3d704d3831ab7d9ea7e927d1c5d816a Jul 23 18:57:16.470236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:16.470183 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:16.655196 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:16.655156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" event={"ID":"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6","Type":"ContainerStarted","Data":"726fef8f21585ebf3046ec97da38f513ee84e172927078fe8dd34e1a67bb5116"} Jul 23 18:57:16.657097 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:16.657070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" event={"ID":"df1b4903-7d42-45e3-8b9f-669671691d8f","Type":"ContainerStarted","Data":"829daa69f13a6e8d65fe89102c99e213b3d704d3831ab7d9ea7e927d1c5d816a"} Jul 23 18:57:16.664781 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:16.664749 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:20.015535 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.015493 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:57:20.015933 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.015566 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:57:20.686426 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.686382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" event={"ID":"01252b47-6b6d-44ee-95c7-57d8e0fd1615","Type":"ContainerStarted","Data":"9d3f6ee63db9d1bfe65a24d5542b8c9426a76169cc946f5b7801722c8a10c7ef"} Jul 23 18:57:20.686626 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.686533 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:20.688277 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.688256 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" event={"ID":"2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6","Type":"ContainerStarted","Data":"12f04fd169b80f3c24b77803fd1fa7baeb5c1d8ecba8f6bd08566720f773a1fd"} Jul 23 18:57:20.688399 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.688340 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:20.689743 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.689721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" event={"ID":"75dc15d6-0580-4360-97f0-a049f8851c2e","Type":"ContainerStarted","Data":"17e53b0755a60257cf462585d7bb85536fabe96dea8e7c79379f7bf96d25a78a"} Jul 23 18:57:20.697498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.697474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" event={"ID":"df1b4903-7d42-45e3-8b9f-669671691d8f","Type":"ContainerStarted","Data":"b6185fde428561797475dab5a4d8a0677043056198f79947c78cb6407c7cc207"} Jul 23 18:57:20.697613 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.697546 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:20.709012 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.708938 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" podStartSLOduration=2.600064059 podStartE2EDuration="7.70892497s" totalImagesPullingTime="5.108860911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:20.70774747 +0000 UTC m=+393.288891684" watchObservedRunningTime="2026-07-23 18:57:20.70892497 +0000 UTC m=+393.290069230" Jul 23 18:57:20.734498 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.734452 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8f784977d-7fbz7" podStartSLOduration=2.474498091 podStartE2EDuration="7.734436258s" totalImagesPullingTime="5.259938167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:20.733088185 +0000 UTC m=+393.314232405" watchObservedRunningTime="2026-07-23 18:57:20.734436258 +0000 UTC m=+393.315580560" Jul 23 18:57:20.768549 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.768495 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" podStartSLOduration=1.626332047 podStartE2EDuration="5.768475544s" totalImagesPullingTime="4.142143497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:20.766608753 +0000 UTC m=+393.347752972" watchObservedRunningTime="2026-07-23 18:57:20.768475544 +0000 UTC m=+393.349619765" Jul 23 18:57:20.816218 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:20.816156 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" podStartSLOduration=2.600608432 podStartE2EDuration="6.816139613s" totalImagesPullingTime="4.215531181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:20.80880913 +0000 UTC m=+393.389953350" watchObservedRunningTime="2026-07-23 18:57:20.816139613 +0000 UTC m=+393.397283834" Jul 23 18:57:21.045190 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.045075 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:21.140320 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.140281 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:21.703975 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.703945 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-f2gf8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 18:57:21.704110 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.703996 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" podUID="df1b4903-7d42-45e3-8b9f-669671691d8f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 18:57:21.861755 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.861718 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj"] Jul 23 18:57:21.868631 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.868610 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.878116 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.878094 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-h85hs\"" Jul 23 18:57:21.948890 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.948862 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.948903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.948925 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.948973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.948994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.949023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.949090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf4np\" (UniqueName: \"kubernetes.io/projected/027ae2f8-e98c-4668-870a-90d35ce919ed-kube-api-access-bf4np\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.949133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/027ae2f8-e98c-4668-870a-90d35ce919ed-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.949276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.949167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:21.965040 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:21.964959 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:22.047815 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.047770 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj"] Jul 23 18:57:22.050040 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/027ae2f8-e98c-4668-870a-90d35ce919ed-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.050659 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.050759 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050702 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.050759 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050735 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.050890 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.050890 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/027ae2f8-e98c-4668-870a-90d35ce919ed-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.050890 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050909 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.050989 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.051000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf4np\" (UniqueName: \"kubernetes.io/projected/027ae2f8-e98c-4668-870a-90d35ce919ed-kube-api-access-bf4np\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051304 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.051202 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051360 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.051308 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.051360 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.051345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.053292 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.053268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.053515 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.053497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.080549 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.080526 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf4np\" (UniqueName: \"kubernetes.io/projected/027ae2f8-e98c-4668-870a-90d35ce919ed-kube-api-access-bf4np\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.111703 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.111669 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/027ae2f8-e98c-4668-870a-90d35ce919ed-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48svgj\" (UID: \"027ae2f8-e98c-4668-870a-90d35ce919ed\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.178585 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.178553 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:22.375364 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.375329 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:22.414369 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:22.414340 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027ae2f8_e98c_4668_870a_90d35ce919ed.slice/crio-8c8cb6aa816867e26913ec01dea8f94e7ed564f59c54ad818dcd459bce35ef46 WatchSource:0}: Error finding container 8c8cb6aa816867e26913ec01dea8f94e7ed564f59c54ad818dcd459bce35ef46: Status 404 returned error can't find the container with id 8c8cb6aa816867e26913ec01dea8f94e7ed564f59c54ad818dcd459bce35ef46 Jul 23 18:57:22.418179 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.418149 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj"] Jul 23 18:57:22.538488 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.538410 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:22.707803 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:22.707772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" event={"ID":"027ae2f8-e98c-4668-870a-90d35ce919ed","Type":"ContainerStarted","Data":"8c8cb6aa816867e26913ec01dea8f94e7ed564f59c54ad818dcd459bce35ef46"} Jul 23 18:57:24.680552 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:24.680511 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:24.703447 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:24.703422 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:24.816813 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:24.816777 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.311756 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.311720 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.412696 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.412661 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.460792 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.460763 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv"] Jul 23 18:57:25.464077 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.464060 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.468194 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.468177 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 18:57:25.468317 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.468183 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jv6bv\"" Jul 23 18:57:25.478099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.478078 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv"] Jul 23 18:57:25.487295 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.487274 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz"] Jul 23 18:57:25.490237 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.490218 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.494013 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.493997 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 23 18:57:25.495988 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.495965 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-m7mxr\"" Jul 23 18:57:25.508118 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.508099 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz"] Jul 23 18:57:25.511283 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.511255 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:57:25.511357 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.511313 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:57:25.511357 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.511353 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:57:25.530381 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.530358 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.584189 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.584164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ctxn\" (UniqueName: \"kubernetes.io/projected/a644b371-42b3-4fb0-a9c8-4ff7c2f71762-kube-api-access-4ctxn\") pod \"llmisvc-controller-manager-777c6bd7d5-jrfwz\" (UID: \"a644b371-42b3-4fb0-a9c8-4ff7c2f71762\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.584290 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.584205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb4q4\" (UniqueName: \"kubernetes.io/projected/3dcf84a9-0697-4547-a309-db90a958dca6-kube-api-access-qb4q4\") pod \"kserve-controller-manager-d6f4b65b8-dtwnv\" (UID: \"3dcf84a9-0697-4547-a309-db90a958dca6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.584290 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.584226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a644b371-42b3-4fb0-a9c8-4ff7c2f71762-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jrfwz\" (UID: \"a644b371-42b3-4fb0-a9c8-4ff7c2f71762\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.584356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.584342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dcf84a9-0697-4547-a309-db90a958dca6-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtwnv\" (UID: \"3dcf84a9-0697-4547-a309-db90a958dca6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.642760 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.642725 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.685859 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.685788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ctxn\" (UniqueName: \"kubernetes.io/projected/a644b371-42b3-4fb0-a9c8-4ff7c2f71762-kube-api-access-4ctxn\") pod \"llmisvc-controller-manager-777c6bd7d5-jrfwz\" (UID: \"a644b371-42b3-4fb0-a9c8-4ff7c2f71762\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.685859 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.685854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb4q4\" (UniqueName: \"kubernetes.io/projected/3dcf84a9-0697-4547-a309-db90a958dca6-kube-api-access-qb4q4\") pod \"kserve-controller-manager-d6f4b65b8-dtwnv\" (UID: \"3dcf84a9-0697-4547-a309-db90a958dca6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.686283 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.685878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a644b371-42b3-4fb0-a9c8-4ff7c2f71762-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jrfwz\" (UID: \"a644b371-42b3-4fb0-a9c8-4ff7c2f71762\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.686283 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.685943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dcf84a9-0697-4547-a309-db90a958dca6-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtwnv\" (UID: \"3dcf84a9-0697-4547-a309-db90a958dca6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.688583 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.688552 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a644b371-42b3-4fb0-a9c8-4ff7c2f71762-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jrfwz\" (UID: \"a644b371-42b3-4fb0-a9c8-4ff7c2f71762\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.688692 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.688558 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dcf84a9-0697-4547-a309-db90a958dca6-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtwnv\" (UID: \"3dcf84a9-0697-4547-a309-db90a958dca6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.703106 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.703081 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb4q4\" (UniqueName: \"kubernetes.io/projected/3dcf84a9-0697-4547-a309-db90a958dca6-kube-api-access-qb4q4\") pod \"kserve-controller-manager-d6f4b65b8-dtwnv\" (UID: \"3dcf84a9-0697-4547-a309-db90a958dca6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.706079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.706054 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ctxn\" (UniqueName: \"kubernetes.io/projected/a644b371-42b3-4fb0-a9c8-4ff7c2f71762-kube-api-access-4ctxn\") pod \"llmisvc-controller-manager-777c6bd7d5-jrfwz\" (UID: \"a644b371-42b3-4fb0-a9c8-4ff7c2f71762\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.726088 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.726064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" event={"ID":"027ae2f8-e98c-4668-870a-90d35ce919ed","Type":"ContainerStarted","Data":"d6b6a49d782fafb8814fed093a3d640bc68ab5e3b5bfb431631611affa821172"} Jul 23 18:57:25.774861 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.774809 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:25.782371 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.782329 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" podStartSLOduration=1.689343489 podStartE2EDuration="4.782309158s" totalImagesPullingTime="3.092965669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:25.780985892 +0000 UTC m=+398.362130111" watchObservedRunningTime="2026-07-23 18:57:25.782309158 +0000 UTC m=+398.363453379" Jul 23 18:57:25.801327 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.801298 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:25.804605 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.804581 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.916907 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.916875 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:25.946158 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.946132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv"] Jul 23 18:57:25.948097 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:25.948062 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcf84a9_0697_4547_a309_db90a958dca6.slice/crio-74ed1858aec0786c4f4c24cd7a700aaa5c0740080d3b9b78f5b10264cf3642dc WatchSource:0}: Error finding container 74ed1858aec0786c4f4c24cd7a700aaa5c0740080d3b9b78f5b10264cf3642dc: Status 404 returned error can't find the container with id 74ed1858aec0786c4f4c24cd7a700aaa5c0740080d3b9b78f5b10264cf3642dc Jul 23 18:57:25.978216 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:25.978180 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz"] Jul 23 18:57:25.981708 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:57:25.981686 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda644b371_42b3_4fb0_a9c8_4ff7c2f71762.slice/crio-df607deaba92ebb3c7976e4e114a71b8f5d1e8d35c4d9798215522d5b1e25bf9 WatchSource:0}: Error finding container df607deaba92ebb3c7976e4e114a71b8f5d1e8d35c4d9798215522d5b1e25bf9: Status 404 returned error can't find the container with id df607deaba92ebb3c7976e4e114a71b8f5d1e8d35c4d9798215522d5b1e25bf9 Jul 23 18:57:26.179766 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:26.179681 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:26.185508 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:26.185481 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:26.732099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:26.732067 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" event={"ID":"3dcf84a9-0697-4547-a309-db90a958dca6","Type":"ContainerStarted","Data":"74ed1858aec0786c4f4c24cd7a700aaa5c0740080d3b9b78f5b10264cf3642dc"} Jul 23 18:57:26.733126 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:26.733105 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" event={"ID":"a644b371-42b3-4fb0-a9c8-4ff7c2f71762","Type":"ContainerStarted","Data":"df607deaba92ebb3c7976e4e114a71b8f5d1e8d35c4d9798215522d5b1e25bf9"} Jul 23 18:57:26.733343 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:26.733325 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:26.734282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:26.734261 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48svgj" Jul 23 18:57:27.824098 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:27.824068 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8"] Jul 23 18:57:27.824553 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:27.824292 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" podUID="df1b4903-7d42-45e3-8b9f-669671691d8f" containerName="discovery" containerID="cri-o://b6185fde428561797475dab5a4d8a0677043056198f79947c78cb6407c7cc207" gracePeriod=30 Jul 23 18:57:28.746244 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:28.746167 2578 generic.go:356] "Generic (PLEG): container finished" podID="df1b4903-7d42-45e3-8b9f-669671691d8f" containerID="b6185fde428561797475dab5a4d8a0677043056198f79947c78cb6407c7cc207" exitCode=0 Jul 23 18:57:28.746397 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:28.746252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" event={"ID":"df1b4903-7d42-45e3-8b9f-669671691d8f","Type":"ContainerDied","Data":"b6185fde428561797475dab5a4d8a0677043056198f79947c78cb6407c7cc207"} Jul 23 18:57:28.996448 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:28.996403 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:29.119225 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119194 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-ca-configmap\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119240 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-kubeconfig\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119260 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-dns-cert\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119465 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrbt5\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-kube-api-access-rrbt5\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119525 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119495 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-cacerts\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119579 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119548 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/df1b4903-7d42-45e3-8b9f-669671691d8f-local-certs\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119579 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119574 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-token\") pod \"df1b4903-7d42-45e3-8b9f-669671691d8f\" (UID: \"df1b4903-7d42-45e3-8b9f-669671691d8f\") " Jul 23 18:57:29.119692 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119572 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:57:29.120055 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.119920 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-ca-configmap\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.121889 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.121863 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-cacerts" (OuterVolumeSpecName: "cacerts") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:57:29.121987 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.121869 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:57:29.122040 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.121985 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:57:29.122196 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.122181 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df1b4903-7d42-45e3-8b9f-669671691d8f-local-certs" (OuterVolumeSpecName: "local-certs") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:57:29.122269 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.122191 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:57:29.122269 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.122206 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-kube-api-access-rrbt5" (OuterVolumeSpecName: "kube-api-access-rrbt5") pod "df1b4903-7d42-45e3-8b9f-669671691d8f" (UID: "df1b4903-7d42-45e3-8b9f-669671691d8f"). InnerVolumeSpecName "kube-api-access-rrbt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:57:29.221213 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.221181 2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-cacerts\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.221213 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.221208 2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/df1b4903-7d42-45e3-8b9f-669671691d8f-local-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.221213 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.221221 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.221418 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.221233 2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-kubeconfig\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.221418 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.221244 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/df1b4903-7d42-45e3-8b9f-669671691d8f-istio-csr-dns-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.221418 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.221256 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrbt5\" (UniqueName: \"kubernetes.io/projected/df1b4903-7d42-45e3-8b9f-669671691d8f-kube-api-access-rrbt5\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:57:29.752598 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.752502 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" event={"ID":"3dcf84a9-0697-4547-a309-db90a958dca6","Type":"ContainerStarted","Data":"25bc6d0675ade92d93729d80c00b4397af614df21660a38a6af1a443ec8df029"} Jul 23 18:57:29.752598 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.752568 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:57:29.753776 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.753756 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" Jul 23 18:57:29.753913 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.753785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8" event={"ID":"df1b4903-7d42-45e3-8b9f-669671691d8f","Type":"ContainerDied","Data":"829daa69f13a6e8d65fe89102c99e213b3d704d3831ab7d9ea7e927d1c5d816a"} Jul 23 18:57:29.753913 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.753848 2578 scope.go:117] "RemoveContainer" containerID="b6185fde428561797475dab5a4d8a0677043056198f79947c78cb6407c7cc207" Jul 23 18:57:29.755285 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.755264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" event={"ID":"a644b371-42b3-4fb0-a9c8-4ff7c2f71762","Type":"ContainerStarted","Data":"7ddc10c66572e8adab3f33a7a3d3835c976bbdf4afdd84691df0757822c9cd14"} Jul 23 18:57:29.755405 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.755392 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:57:29.777209 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.777168 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" podStartSLOduration=1.756198866 podStartE2EDuration="4.777153252s" totalImagesPullingTime="3.020954386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:29.774902329 +0000 UTC m=+402.356046549" watchObservedRunningTime="2026-07-23 18:57:29.777153252 +0000 UTC m=+402.358297472" Jul 23 18:57:29.795627 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.795603 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8"] Jul 23 18:57:29.800058 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.800036 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-f2gf8"] Jul 23 18:57:29.825352 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.825312 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" podStartSLOduration=1.785058635 podStartE2EDuration="4.825300653s" totalImagesPullingTime="3.040242018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:57:29.825046556 +0000 UTC m=+402.406190775" watchObservedRunningTime="2026-07-23 18:57:29.825300653 +0000 UTC m=+402.406444886" Jul 23 18:57:29.922742 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:29.922717 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df1b4903-7d42-45e3-8b9f-669671691d8f" path="/var/lib/kubelet/pods/df1b4903-7d42-45e3-8b9f-669671691d8f/volumes" Jul 23 18:57:31.704539 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:31.704516 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nqrct" Jul 23 18:57:31.707175 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:31.707151 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-29h4l" Jul 23 18:57:34.063427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:34.063382 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:34.129385 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:34.129359 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:57:50.796606 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:57:50.796567 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:00.763773 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:00.763744 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jrfwz" Jul 23 18:58:00.766967 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:00.766948 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtwnv" Jul 23 18:58:09.018668 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.018590 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:09.117440 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.117411 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:09.159845 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.159793 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-998ck"] Jul 23 18:58:09.160180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.160166 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df1b4903-7d42-45e3-8b9f-669671691d8f" containerName="discovery" Jul 23 18:58:09.160180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.160182 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="df1b4903-7d42-45e3-8b9f-669671691d8f" containerName="discovery" Jul 23 18:58:09.160271 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.160245 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="df1b4903-7d42-45e3-8b9f-669671691d8f" containerName="discovery" Jul 23 18:58:09.162006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.161991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:09.164930 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.164905 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 18:58:09.165681 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.165661 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 18:58:09.165793 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.165736 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kfckw\"" Jul 23 18:58:09.174996 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.174974 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-998ck"] Jul 23 18:58:09.258669 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.258635 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:09.260804 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.260779 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5chc\" (UniqueName: \"kubernetes.io/projected/802898a2-f73c-4021-8ff6-5f053d015653-kube-api-access-k5chc\") pod \"kuadrant-operator-catalog-998ck\" (UID: \"802898a2-f73c-4021-8ff6-5f053d015653\") " pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:09.361777 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.361745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5chc\" (UniqueName: \"kubernetes.io/projected/802898a2-f73c-4021-8ff6-5f053d015653-kube-api-access-k5chc\") pod \"kuadrant-operator-catalog-998ck\" (UID: \"802898a2-f73c-4021-8ff6-5f053d015653\") " pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:09.370440 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.370415 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5chc\" (UniqueName: \"kubernetes.io/projected/802898a2-f73c-4021-8ff6-5f053d015653-kube-api-access-k5chc\") pod \"kuadrant-operator-catalog-998ck\" (UID: \"802898a2-f73c-4021-8ff6-5f053d015653\") " pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:09.472629 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.472597 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:09.509071 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.509040 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-998ck"] Jul 23 18:58:09.606689 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.606665 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-998ck"] Jul 23 18:58:09.608975 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:09.608951 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802898a2_f73c_4021_8ff6_5f053d015653.slice/crio-d11710685f280beba2444a7a0b51ab4fee6cfa48b9dfe950c606f535b213fab6 WatchSource:0}: Error finding container d11710685f280beba2444a7a0b51ab4fee6cfa48b9dfe950c606f535b213fab6: Status 404 returned error can't find the container with id d11710685f280beba2444a7a0b51ab4fee6cfa48b9dfe950c606f535b213fab6 Jul 23 18:58:09.721771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.721741 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vz8fd"] Jul 23 18:58:09.724464 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.724448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:09.733678 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.733654 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vz8fd"] Jul 23 18:58:09.765347 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.765322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rzxq\" (UniqueName: \"kubernetes.io/projected/77beb20c-6335-4565-ae02-e4976d548945-kube-api-access-6rzxq\") pod \"kuadrant-operator-catalog-vz8fd\" (UID: \"77beb20c-6335-4565-ae02-e4976d548945\") " pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:09.866733 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.866641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rzxq\" (UniqueName: \"kubernetes.io/projected/77beb20c-6335-4565-ae02-e4976d548945-kube-api-access-6rzxq\") pod \"kuadrant-operator-catalog-vz8fd\" (UID: \"77beb20c-6335-4565-ae02-e4976d548945\") " pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:09.875646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.875619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rzxq\" (UniqueName: \"kubernetes.io/projected/77beb20c-6335-4565-ae02-e4976d548945-kube-api-access-6rzxq\") pod \"kuadrant-operator-catalog-vz8fd\" (UID: \"77beb20c-6335-4565-ae02-e4976d548945\") " pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:09.944848 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:09.944803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-998ck" event={"ID":"802898a2-f73c-4021-8ff6-5f053d015653","Type":"ContainerStarted","Data":"d11710685f280beba2444a7a0b51ab4fee6cfa48b9dfe950c606f535b213fab6"} Jul 23 18:58:10.034615 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:10.034581 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:10.169183 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:10.169158 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vz8fd"] Jul 23 18:58:10.171693 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:10.171661 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77beb20c_6335_4565_ae02_e4976d548945.slice/crio-6fd4c6bce0f968fa6b2478cef782ac318eb065999c87abd8a7aa08e2af92b92a WatchSource:0}: Error finding container 6fd4c6bce0f968fa6b2478cef782ac318eb065999c87abd8a7aa08e2af92b92a: Status 404 returned error can't find the container with id 6fd4c6bce0f968fa6b2478cef782ac318eb065999c87abd8a7aa08e2af92b92a Jul 23 18:58:10.952906 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:10.952815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" event={"ID":"77beb20c-6335-4565-ae02-e4976d548945","Type":"ContainerStarted","Data":"6fd4c6bce0f968fa6b2478cef782ac318eb065999c87abd8a7aa08e2af92b92a"} Jul 23 18:58:11.958751 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:11.958662 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" event={"ID":"77beb20c-6335-4565-ae02-e4976d548945","Type":"ContainerStarted","Data":"75d2d5b770b1f849cdde696642813a26604e4ec4e7d356995674d8e564014a68"} Jul 23 18:58:11.960258 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:11.960236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-998ck" event={"ID":"802898a2-f73c-4021-8ff6-5f053d015653","Type":"ContainerStarted","Data":"5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c"} Jul 23 18:58:11.960362 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:11.960314 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-998ck" podUID="802898a2-f73c-4021-8ff6-5f053d015653" containerName="registry-server" containerID="cri-o://5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c" gracePeriod=2 Jul 23 18:58:11.977804 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:11.977762 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" podStartSLOduration=1.444639955 podStartE2EDuration="2.977749422s" totalImagesPullingTime="1.533109467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:58:11.975787122 +0000 UTC m=+444.556931342" watchObservedRunningTime="2026-07-23 18:58:11.977749422 +0000 UTC m=+444.558893642" Jul 23 18:58:11.995387 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:11.995352 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-998ck" podStartSLOduration=0.90270468 podStartE2EDuration="2.995340169s" totalImagesPullingTime="2.092635489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:58:11.994629964 +0000 UTC m=+444.575774165" watchObservedRunningTime="2026-07-23 18:58:11.995340169 +0000 UTC m=+444.576484389" Jul 23 18:58:12.197490 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.197468 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:12.286465 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.286379 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5chc\" (UniqueName: \"kubernetes.io/projected/802898a2-f73c-4021-8ff6-5f053d015653-kube-api-access-k5chc\") pod \"802898a2-f73c-4021-8ff6-5f053d015653\" (UID: \"802898a2-f73c-4021-8ff6-5f053d015653\") " Jul 23 18:58:12.288593 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.288569 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/802898a2-f73c-4021-8ff6-5f053d015653-kube-api-access-k5chc" (OuterVolumeSpecName: "kube-api-access-k5chc") pod "802898a2-f73c-4021-8ff6-5f053d015653" (UID: "802898a2-f73c-4021-8ff6-5f053d015653"). InnerVolumeSpecName "kube-api-access-k5chc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:58:12.387137 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.387107 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5chc\" (UniqueName: \"kubernetes.io/projected/802898a2-f73c-4021-8ff6-5f053d015653-kube-api-access-k5chc\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:12.966469 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.966432 2578 generic.go:356] "Generic (PLEG): container finished" podID="802898a2-f73c-4021-8ff6-5f053d015653" containerID="5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c" exitCode=0 Jul 23 18:58:12.966469 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.966473 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-998ck" Jul 23 18:58:12.966975 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.966526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-998ck" event={"ID":"802898a2-f73c-4021-8ff6-5f053d015653","Type":"ContainerDied","Data":"5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c"} Jul 23 18:58:12.966975 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.966571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-998ck" event={"ID":"802898a2-f73c-4021-8ff6-5f053d015653","Type":"ContainerDied","Data":"d11710685f280beba2444a7a0b51ab4fee6cfa48b9dfe950c606f535b213fab6"} Jul 23 18:58:12.966975 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.966595 2578 scope.go:117] "RemoveContainer" containerID="5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c" Jul 23 18:58:12.976978 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.976962 2578 scope.go:117] "RemoveContainer" containerID="5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c" Jul 23 18:58:12.977229 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:58:12.977210 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c\": container with ID starting with 5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c not found: ID does not exist" containerID="5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c" Jul 23 18:58:12.977288 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.977239 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c"} err="failed to get container status \"5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c\": rpc error: code = NotFound desc = could not find container \"5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c\": container with ID starting with 5f837a3fb1425ecb3ff421c29f9fa7ee6631063c04f2def5812e2354c330e47c not found: ID does not exist" Jul 23 18:58:12.989739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.989716 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-998ck"] Jul 23 18:58:12.993869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:12.993847 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-998ck"] Jul 23 18:58:13.922510 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:13.922482 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="802898a2-f73c-4021-8ff6-5f053d015653" path="/var/lib/kubelet/pods/802898a2-f73c-4021-8ff6-5f053d015653/volumes" Jul 23 18:58:20.035644 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:20.035615 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:20.036117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:20.035672 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:20.058575 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:20.058539 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:21.026638 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:21.026613 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vz8fd" Jul 23 18:58:24.555089 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.555051 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp"] Jul 23 18:58:24.555448 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.555384 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="802898a2-f73c-4021-8ff6-5f053d015653" containerName="registry-server" Jul 23 18:58:24.555448 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.555395 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="802898a2-f73c-4021-8ff6-5f053d015653" containerName="registry-server" Jul 23 18:58:24.555516 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.555469 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="802898a2-f73c-4021-8ff6-5f053d015653" containerName="registry-server" Jul 23 18:58:24.560000 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.559982 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.562888 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.562868 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kzh8g\"" Jul 23 18:58:24.571463 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.571442 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp"] Jul 23 18:58:24.590765 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.590741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.590884 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.590784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnnrp\" (UniqueName: \"kubernetes.io/projected/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-kube-api-access-lnnrp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.590884 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.590872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.691646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.691620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.691755 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.691712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.691812 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.691765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnrp\" (UniqueName: \"kubernetes.io/projected/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-kube-api-access-lnnrp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.692096 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.692078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.692135 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.692093 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.702125 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.702095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnrp\" (UniqueName: \"kubernetes.io/projected/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-kube-api-access-lnnrp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:24.869754 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:24.869709 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:25.008896 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.008872 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp"] Jul 23 18:58:25.011122 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:25.011092 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eca061d_f05a_4597_a3c4_387e3ef3b1e3.slice/crio-d0c34fa1cf7b5c55196d289169cafc1de9dfeee1e3f50dd74934379e362946c4 WatchSource:0}: Error finding container d0c34fa1cf7b5c55196d289169cafc1de9dfeee1e3f50dd74934379e362946c4: Status 404 returned error can't find the container with id d0c34fa1cf7b5c55196d289169cafc1de9dfeee1e3f50dd74934379e362946c4 Jul 23 18:58:25.022375 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.022350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" event={"ID":"7eca061d-f05a-4597-a3c4-387e3ef3b1e3","Type":"ContainerStarted","Data":"d0c34fa1cf7b5c55196d289169cafc1de9dfeee1e3f50dd74934379e362946c4"} Jul 23 18:58:25.164921 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.164856 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk"] Jul 23 18:58:25.168371 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.168354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.182599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.182570 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk"] Jul 23 18:58:25.297633 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.297592 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.297789 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.297647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttlh\" (UniqueName: \"kubernetes.io/projected/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-kube-api-access-lttlh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.297789 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.297706 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.399015 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.398981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.399196 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.399039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.399196 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.399064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttlh\" (UniqueName: \"kubernetes.io/projected/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-kube-api-access-lttlh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.399346 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.399325 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.399410 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.399381 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.409721 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.409700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttlh\" (UniqueName: \"kubernetes.io/projected/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-kube-api-access-lttlh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.484764 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.484689 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:25.547805 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.547771 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld"] Jul 23 18:58:25.552506 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.552485 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.562324 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.562301 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld"] Jul 23 18:58:25.601386 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.601356 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.601549 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.601422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66bnl\" (UniqueName: \"kubernetes.io/projected/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-kube-api-access-66bnl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.601549 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.601471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.622882 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.622852 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk"] Jul 23 18:58:25.625787 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:25.625763 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc746bf6c_3ec6_4501_b7ba_5a54c4f75598.slice/crio-e2c42b3c4e6e2126acdd935c33998db5fe18bbd3e657640d46b10a2c9dddb32f WatchSource:0}: Error finding container e2c42b3c4e6e2126acdd935c33998db5fe18bbd3e657640d46b10a2c9dddb32f: Status 404 returned error can't find the container with id e2c42b3c4e6e2126acdd935c33998db5fe18bbd3e657640d46b10a2c9dddb32f Jul 23 18:58:25.702555 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.702312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.702555 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.702405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66bnl\" (UniqueName: \"kubernetes.io/projected/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-kube-api-access-66bnl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.702555 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.702464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.702846 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.702801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.702951 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.702873 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.713431 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.713404 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66bnl\" (UniqueName: \"kubernetes.io/projected/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-kube-api-access-66bnl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.863996 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.863963 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:25.988183 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.988118 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz"] Jul 23 18:58:25.992976 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:25.992961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.009236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.009213 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz"] Jul 23 18:58:26.009811 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:26.009788 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7718953_8cc3_4cba_acef_ec0b1d1fd4b7.slice/crio-55ab20ae208c5635b23e5b0a8b9f0db4ec37a3666121513e15ee51f2c7da097e WatchSource:0}: Error finding container 55ab20ae208c5635b23e5b0a8b9f0db4ec37a3666121513e15ee51f2c7da097e: Status 404 returned error can't find the container with id 55ab20ae208c5635b23e5b0a8b9f0db4ec37a3666121513e15ee51f2c7da097e Jul 23 18:58:26.017061 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.012511 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld"] Jul 23 18:58:26.028974 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.028950 2578 generic.go:356] "Generic (PLEG): container finished" podID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerID="6b77489ca432c0540ad41fc2de0f419dc2c68cd61a8faa63fe668fdc944efffe" exitCode=0 Jul 23 18:58:26.029075 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.028987 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" event={"ID":"c746bf6c-3ec6-4501-b7ba-5a54c4f75598","Type":"ContainerDied","Data":"6b77489ca432c0540ad41fc2de0f419dc2c68cd61a8faa63fe668fdc944efffe"} Jul 23 18:58:26.029075 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.029038 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" event={"ID":"c746bf6c-3ec6-4501-b7ba-5a54c4f75598","Type":"ContainerStarted","Data":"e2c42b3c4e6e2126acdd935c33998db5fe18bbd3e657640d46b10a2c9dddb32f"} Jul 23 18:58:26.030383 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.030361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" event={"ID":"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7","Type":"ContainerStarted","Data":"55ab20ae208c5635b23e5b0a8b9f0db4ec37a3666121513e15ee51f2c7da097e"} Jul 23 18:58:26.031903 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.031878 2578 generic.go:356] "Generic (PLEG): container finished" podID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerID="0fe5c751dee9f42a2bbfd534a5cae534d1f5bb354e1bb95cce1777b0bf843c18" exitCode=0 Jul 23 18:58:26.031996 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.031918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" event={"ID":"7eca061d-f05a-4597-a3c4-387e3ef3b1e3","Type":"ContainerDied","Data":"0fe5c751dee9f42a2bbfd534a5cae534d1f5bb354e1bb95cce1777b0bf843c18"} Jul 23 18:58:26.105267 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.105236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vxxp\" (UniqueName: \"kubernetes.io/projected/658de733-506e-41ff-9acb-ca3b818424c8-kube-api-access-7vxxp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.105410 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.105276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.105410 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.105302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.205965 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.205937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.206098 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.205979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.206098 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.206063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vxxp\" (UniqueName: \"kubernetes.io/projected/658de733-506e-41ff-9acb-ca3b818424c8-kube-api-access-7vxxp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.206321 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.206301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.206370 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.206312 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.214597 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.214578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vxxp\" (UniqueName: \"kubernetes.io/projected/658de733-506e-41ff-9acb-ca3b818424c8-kube-api-access-7vxxp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.302042 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.301966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:26.435906 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.435885 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz"] Jul 23 18:58:26.437961 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:26.437931 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658de733_506e_41ff_9acb_ca3b818424c8.slice/crio-41f2fe33aff9108ce7338e3d6ee5791fe331ee5911afdfa2f8c9886566efb89e WatchSource:0}: Error finding container 41f2fe33aff9108ce7338e3d6ee5791fe331ee5911afdfa2f8c9886566efb89e: Status 404 returned error can't find the container with id 41f2fe33aff9108ce7338e3d6ee5791fe331ee5911afdfa2f8c9886566efb89e Jul 23 18:58:26.942683 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:26.942650 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:27.009003 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.008974 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:27.037403 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.037383 2578 generic.go:356] "Generic (PLEG): container finished" podID="658de733-506e-41ff-9acb-ca3b818424c8" containerID="14aba4adfc091c689db977925796f74191e179e96b880bfa9b4856647c464b40" exitCode=0 Jul 23 18:58:27.037500 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.037474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" event={"ID":"658de733-506e-41ff-9acb-ca3b818424c8","Type":"ContainerDied","Data":"14aba4adfc091c689db977925796f74191e179e96b880bfa9b4856647c464b40"} Jul 23 18:58:27.037558 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.037523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" event={"ID":"658de733-506e-41ff-9acb-ca3b818424c8","Type":"ContainerStarted","Data":"41f2fe33aff9108ce7338e3d6ee5791fe331ee5911afdfa2f8c9886566efb89e"} Jul 23 18:58:27.039095 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.039075 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerID="3d1d989701d88f2c5824ad4f24c52048e47d3ae4e75c26ad0af4ac64c785ea9a" exitCode=0 Jul 23 18:58:27.039190 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.039160 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" event={"ID":"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7","Type":"ContainerDied","Data":"3d1d989701d88f2c5824ad4f24c52048e47d3ae4e75c26ad0af4ac64c785ea9a"} Jul 23 18:58:27.041198 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.041173 2578 generic.go:356] "Generic (PLEG): container finished" podID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerID="90f8e975b3ba47ce43944243014221a99272bfc665fc81cde702432b29edd0d0" exitCode=0 Jul 23 18:58:27.041295 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:27.041205 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" event={"ID":"7eca061d-f05a-4597-a3c4-387e3ef3b1e3","Type":"ContainerDied","Data":"90f8e975b3ba47ce43944243014221a99272bfc665fc81cde702432b29edd0d0"} Jul 23 18:58:28.049926 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:28.049897 2578 generic.go:356] "Generic (PLEG): container finished" podID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerID="25680c6c38f9dccb8cf6d378f967bd30a6336a3c918d76eb45e6f57510726ec0" exitCode=0 Jul 23 18:58:28.050357 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:28.049980 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" event={"ID":"7eca061d-f05a-4597-a3c4-387e3ef3b1e3","Type":"ContainerDied","Data":"25680c6c38f9dccb8cf6d378f967bd30a6336a3c918d76eb45e6f57510726ec0"} Jul 23 18:58:28.052365 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:28.052334 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" event={"ID":"658de733-506e-41ff-9acb-ca3b818424c8","Type":"ContainerStarted","Data":"0eef760b500962d3b0d70d34c14b912503ba62d6aeac92a99302ee7d8d73115a"} Jul 23 18:58:28.054567 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:28.054534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" event={"ID":"c746bf6c-3ec6-4501-b7ba-5a54c4f75598","Type":"ContainerStarted","Data":"20dc855bf67461fc2d93ccd1db72dfa26201f6158cc8d13085a65434b170770a"} Jul 23 18:58:28.058665 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:28.057908 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" event={"ID":"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7","Type":"ContainerStarted","Data":"9ea1805dd542147def34efc398b737f777117b4829c86b4fd51a36585369fed1"} Jul 23 18:58:29.064795 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.064761 2578 generic.go:356] "Generic (PLEG): container finished" podID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerID="20dc855bf67461fc2d93ccd1db72dfa26201f6158cc8d13085a65434b170770a" exitCode=0 Jul 23 18:58:29.065282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.064853 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" event={"ID":"c746bf6c-3ec6-4501-b7ba-5a54c4f75598","Type":"ContainerDied","Data":"20dc855bf67461fc2d93ccd1db72dfa26201f6158cc8d13085a65434b170770a"} Jul 23 18:58:29.066767 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.066746 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerID="9ea1805dd542147def34efc398b737f777117b4829c86b4fd51a36585369fed1" exitCode=0 Jul 23 18:58:29.066889 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.066849 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" event={"ID":"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7","Type":"ContainerDied","Data":"9ea1805dd542147def34efc398b737f777117b4829c86b4fd51a36585369fed1"} Jul 23 18:58:29.068758 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.068737 2578 generic.go:356] "Generic (PLEG): container finished" podID="658de733-506e-41ff-9acb-ca3b818424c8" containerID="0eef760b500962d3b0d70d34c14b912503ba62d6aeac92a99302ee7d8d73115a" exitCode=0 Jul 23 18:58:29.068866 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.068839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" event={"ID":"658de733-506e-41ff-9acb-ca3b818424c8","Type":"ContainerDied","Data":"0eef760b500962d3b0d70d34c14b912503ba62d6aeac92a99302ee7d8d73115a"} Jul 23 18:58:29.220074 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.220043 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:29.334166 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.334141 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-bundle\") pod \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " Jul 23 18:58:29.334309 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.334215 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnnrp\" (UniqueName: \"kubernetes.io/projected/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-kube-api-access-lnnrp\") pod \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " Jul 23 18:58:29.334309 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.334282 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-util\") pod \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\" (UID: \"7eca061d-f05a-4597-a3c4-387e3ef3b1e3\") " Jul 23 18:58:29.334861 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.334814 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-bundle" (OuterVolumeSpecName: "bundle") pod "7eca061d-f05a-4597-a3c4-387e3ef3b1e3" (UID: "7eca061d-f05a-4597-a3c4-387e3ef3b1e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:29.336440 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.336404 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-kube-api-access-lnnrp" (OuterVolumeSpecName: "kube-api-access-lnnrp") pod "7eca061d-f05a-4597-a3c4-387e3ef3b1e3" (UID: "7eca061d-f05a-4597-a3c4-387e3ef3b1e3"). InnerVolumeSpecName "kube-api-access-lnnrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:58:29.339180 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.339149 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-util" (OuterVolumeSpecName: "util") pod "7eca061d-f05a-4597-a3c4-387e3ef3b1e3" (UID: "7eca061d-f05a-4597-a3c4-387e3ef3b1e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:29.435427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.435394 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:29.435427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.435424 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:29.435427 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:29.435433 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnnrp\" (UniqueName: \"kubernetes.io/projected/7eca061d-f05a-4597-a3c4-387e3ef3b1e3-kube-api-access-lnnrp\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:30.075520 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.075485 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerID="ea31dc8e64deb33430bdf11fee8ae802bb2cd82301d718d6696b5f720a71eedd" exitCode=0 Jul 23 18:58:30.075919 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.075574 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" event={"ID":"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7","Type":"ContainerDied","Data":"ea31dc8e64deb33430bdf11fee8ae802bb2cd82301d718d6696b5f720a71eedd"} Jul 23 18:58:30.077479 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.077460 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" Jul 23 18:58:30.077579 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.077478 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp" event={"ID":"7eca061d-f05a-4597-a3c4-387e3ef3b1e3","Type":"ContainerDied","Data":"d0c34fa1cf7b5c55196d289169cafc1de9dfeee1e3f50dd74934379e362946c4"} Jul 23 18:58:30.077579 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.077523 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0c34fa1cf7b5c55196d289169cafc1de9dfeee1e3f50dd74934379e362946c4" Jul 23 18:58:30.079890 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.079863 2578 generic.go:356] "Generic (PLEG): container finished" podID="658de733-506e-41ff-9acb-ca3b818424c8" containerID="b74613e2d07728378aee6da75bab390a36f61e93bd7a92da8ce64d986e2bfff2" exitCode=0 Jul 23 18:58:30.080004 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.079946 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" event={"ID":"658de733-506e-41ff-9acb-ca3b818424c8","Type":"ContainerDied","Data":"b74613e2d07728378aee6da75bab390a36f61e93bd7a92da8ce64d986e2bfff2"} Jul 23 18:58:30.081905 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.081882 2578 generic.go:356] "Generic (PLEG): container finished" podID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerID="f2dd4277050d936ef8f28f4681c61776f0e6eaaea2218b1a2ea59f233ce57666" exitCode=0 Jul 23 18:58:30.081983 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:30.081927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" event={"ID":"c746bf6c-3ec6-4501-b7ba-5a54c4f75598","Type":"ContainerDied","Data":"f2dd4277050d936ef8f28f4681c61776f0e6eaaea2218b1a2ea59f233ce57666"} Jul 23 18:58:31.242082 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.242053 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:31.284199 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.284174 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:31.288803 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.288784 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:31.350510 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350413 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-util\") pod \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " Jul 23 18:58:31.350510 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350485 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lttlh\" (UniqueName: \"kubernetes.io/projected/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-kube-api-access-lttlh\") pod \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " Jul 23 18:58:31.350739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350546 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-util\") pod \"658de733-506e-41ff-9acb-ca3b818424c8\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " Jul 23 18:58:31.350739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350597 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66bnl\" (UniqueName: \"kubernetes.io/projected/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-kube-api-access-66bnl\") pod \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " Jul 23 18:58:31.350739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350632 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-bundle\") pod \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " Jul 23 18:58:31.350739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350658 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-bundle\") pod \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\" (UID: \"c746bf6c-3ec6-4501-b7ba-5a54c4f75598\") " Jul 23 18:58:31.350739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350701 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vxxp\" (UniqueName: \"kubernetes.io/projected/658de733-506e-41ff-9acb-ca3b818424c8-kube-api-access-7vxxp\") pod \"658de733-506e-41ff-9acb-ca3b818424c8\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " Jul 23 18:58:31.350739 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350731 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-bundle\") pod \"658de733-506e-41ff-9acb-ca3b818424c8\" (UID: \"658de733-506e-41ff-9acb-ca3b818424c8\") " Jul 23 18:58:31.351092 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.350780 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-util\") pod \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\" (UID: \"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7\") " Jul 23 18:58:31.351248 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.351216 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-bundle" (OuterVolumeSpecName: "bundle") pod "e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" (UID: "e7718953-8cc3-4cba-acef-ec0b1d1fd4b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:31.351612 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.351582 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-bundle" (OuterVolumeSpecName: "bundle") pod "c746bf6c-3ec6-4501-b7ba-5a54c4f75598" (UID: "c746bf6c-3ec6-4501-b7ba-5a54c4f75598"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:31.351810 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.351777 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-bundle" (OuterVolumeSpecName: "bundle") pod "658de733-506e-41ff-9acb-ca3b818424c8" (UID: "658de733-506e-41ff-9acb-ca3b818424c8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:31.353042 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.353005 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658de733-506e-41ff-9acb-ca3b818424c8-kube-api-access-7vxxp" (OuterVolumeSpecName: "kube-api-access-7vxxp") pod "658de733-506e-41ff-9acb-ca3b818424c8" (UID: "658de733-506e-41ff-9acb-ca3b818424c8"). InnerVolumeSpecName "kube-api-access-7vxxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:58:31.353488 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.353462 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-kube-api-access-lttlh" (OuterVolumeSpecName: "kube-api-access-lttlh") pod "c746bf6c-3ec6-4501-b7ba-5a54c4f75598" (UID: "c746bf6c-3ec6-4501-b7ba-5a54c4f75598"). InnerVolumeSpecName "kube-api-access-lttlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:58:31.353568 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.353457 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-kube-api-access-66bnl" (OuterVolumeSpecName: "kube-api-access-66bnl") pod "e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" (UID: "e7718953-8cc3-4cba-acef-ec0b1d1fd4b7"). InnerVolumeSpecName "kube-api-access-66bnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:58:31.356272 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.356243 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-util" (OuterVolumeSpecName: "util") pod "658de733-506e-41ff-9acb-ca3b818424c8" (UID: "658de733-506e-41ff-9acb-ca3b818424c8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:31.436049 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.436011 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-util" (OuterVolumeSpecName: "util") pod "e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" (UID: "e7718953-8cc3-4cba-acef-ec0b1d1fd4b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:31.441527 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.441493 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-util" (OuterVolumeSpecName: "util") pod "c746bf6c-3ec6-4501-b7ba-5a54c4f75598" (UID: "c746bf6c-3ec6-4501-b7ba-5a54c4f75598"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:58:31.452044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452023 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lttlh\" (UniqueName: \"kubernetes.io/projected/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-kube-api-access-lttlh\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452046 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452056 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66bnl\" (UniqueName: \"kubernetes.io/projected/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-kube-api-access-66bnl\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452066 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452074 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452082 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vxxp\" (UniqueName: \"kubernetes.io/projected/658de733-506e-41ff-9acb-ca3b818424c8-kube-api-access-7vxxp\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452091 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658de733-506e-41ff-9acb-ca3b818424c8-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452099 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7718953-8cc3-4cba-acef-ec0b1d1fd4b7-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:31.452117 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:31.452107 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c746bf6c-3ec6-4501-b7ba-5a54c4f75598-util\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:58:32.099552 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.099519 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" Jul 23 18:58:32.099744 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.099551 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld" event={"ID":"e7718953-8cc3-4cba-acef-ec0b1d1fd4b7","Type":"ContainerDied","Data":"55ab20ae208c5635b23e5b0a8b9f0db4ec37a3666121513e15ee51f2c7da097e"} Jul 23 18:58:32.099744 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.099602 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55ab20ae208c5635b23e5b0a8b9f0db4ec37a3666121513e15ee51f2c7da097e" Jul 23 18:58:32.101798 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.101771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" event={"ID":"658de733-506e-41ff-9acb-ca3b818424c8","Type":"ContainerDied","Data":"41f2fe33aff9108ce7338e3d6ee5791fe331ee5911afdfa2f8c9886566efb89e"} Jul 23 18:58:32.101947 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.101804 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz" Jul 23 18:58:32.102163 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.101811 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41f2fe33aff9108ce7338e3d6ee5791fe331ee5911afdfa2f8c9886566efb89e" Jul 23 18:58:32.103733 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.103710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" event={"ID":"c746bf6c-3ec6-4501-b7ba-5a54c4f75598","Type":"ContainerDied","Data":"e2c42b3c4e6e2126acdd935c33998db5fe18bbd3e657640d46b10a2c9dddb32f"} Jul 23 18:58:32.103880 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.103748 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2c42b3c4e6e2126acdd935c33998db5fe18bbd3e657640d46b10a2c9dddb32f" Jul 23 18:58:32.103880 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:32.103781 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk" Jul 23 18:58:34.699349 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699314 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bf64c7869-vnw4l"] Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699688 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="util" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699700 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="util" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699707 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="pull" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699712 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="pull" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699722 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="util" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699728 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="util" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699736 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="extract" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699741 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="extract" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699749 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="util" Jul 23 18:58:34.699752 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699754 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="util" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699762 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="util" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699767 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="util" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699774 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699778 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699786 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699791 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699800 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="pull" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699805 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="pull" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699810 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="pull" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699814 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="pull" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699836 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699841 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699849 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="pull" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699854 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="pull" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699905 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="658de733-506e-41ff-9acb-ca3b818424c8" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699913 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c746bf6c-3ec6-4501-b7ba-5a54c4f75598" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699919 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7718953-8cc3-4cba-acef-ec0b1d1fd4b7" containerName="extract" Jul 23 18:58:34.700111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.699927 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eca061d-f05a-4597-a3c4-387e3ef3b1e3" containerName="extract" Jul 23 18:58:34.706955 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.706936 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.720729 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.720706 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf64c7869-vnw4l"] Jul 23 18:58:34.760193 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.760158 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:34.782308 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40637513-b377-4170-8927-9982d61d3696-console-serving-cert\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.782473 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40637513-b377-4170-8927-9982d61d3696-console-oauth-config\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.782473 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-oauth-serving-cert\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.782473 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-service-ca\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.782473 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782445 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c7v8\" (UniqueName: \"kubernetes.io/projected/40637513-b377-4170-8927-9982d61d3696-kube-api-access-7c7v8\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.782653 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-console-config\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.782653 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.782578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-trusted-ca-bundle\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.864802 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.864774 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:34.883982 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.883962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40637513-b377-4170-8927-9982d61d3696-console-oauth-config\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.884064 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.883987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-oauth-serving-cert\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.884064 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.884021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-service-ca\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.884064 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.884041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c7v8\" (UniqueName: \"kubernetes.io/projected/40637513-b377-4170-8927-9982d61d3696-kube-api-access-7c7v8\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.884197 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.884074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-console-config\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.884197 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.884128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-trusted-ca-bundle\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.884292 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.884270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40637513-b377-4170-8927-9982d61d3696-console-serving-cert\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.885204 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.885159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-oauth-serving-cert\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.885318 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.885237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-service-ca\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.885388 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.885343 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-console-config\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.885700 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.885672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40637513-b377-4170-8927-9982d61d3696-trusted-ca-bundle\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.887398 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.887377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40637513-b377-4170-8927-9982d61d3696-console-oauth-config\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.892659 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.892636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40637513-b377-4170-8927-9982d61d3696-console-serving-cert\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:34.894625 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:34.894597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c7v8\" (UniqueName: \"kubernetes.io/projected/40637513-b377-4170-8927-9982d61d3696-kube-api-access-7c7v8\") pod \"console-6bf64c7869-vnw4l\" (UID: \"40637513-b377-4170-8927-9982d61d3696\") " pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:35.016257 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:35.016178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:35.160672 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:35.160644 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf64c7869-vnw4l"] Jul 23 18:58:35.163260 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:35.163229 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40637513_b377_4170_8927_9982d61d3696.slice/crio-0de45913d0489b8896b33e6a1558acc4c2b4d660264e53d482610987e663bcde WatchSource:0}: Error finding container 0de45913d0489b8896b33e6a1558acc4c2b4d660264e53d482610987e663bcde: Status 404 returned error can't find the container with id 0de45913d0489b8896b33e6a1558acc4c2b4d660264e53d482610987e663bcde Jul 23 18:58:36.127700 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:36.127663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf64c7869-vnw4l" event={"ID":"40637513-b377-4170-8927-9982d61d3696","Type":"ContainerStarted","Data":"3459d049c294b620f7b414cfa238311a5ab3c25f7f389a7c279c73712b6dd130"} Jul 23 18:58:36.128101 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:36.127723 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf64c7869-vnw4l" event={"ID":"40637513-b377-4170-8927-9982d61d3696","Type":"ContainerStarted","Data":"0de45913d0489b8896b33e6a1558acc4c2b4d660264e53d482610987e663bcde"} Jul 23 18:58:36.149317 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:36.149268 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bf64c7869-vnw4l" podStartSLOduration=2.149252959 podStartE2EDuration="2.149252959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:58:36.147232304 +0000 UTC m=+468.728376523" watchObservedRunningTime="2026-07-23 18:58:36.149252959 +0000 UTC m=+468.730397178" Jul 23 18:58:38.118965 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:38.118904 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:40.697471 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.697440 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h"] Jul 23 18:58:40.703292 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.703274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:40.706022 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.706005 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5mc7z\"" Jul 23 18:58:40.711544 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.711519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h"] Jul 23 18:58:40.834877 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.834814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pd54\" (UniqueName: \"kubernetes.io/projected/f56ae7ab-5ad3-47a1-b0db-535f24a98089-kube-api-access-5pd54\") pod \"limitador-operator-controller-manager-859d7b78bd-zg57h\" (UID: \"f56ae7ab-5ad3-47a1-b0db-535f24a98089\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:40.936022 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.935984 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pd54\" (UniqueName: \"kubernetes.io/projected/f56ae7ab-5ad3-47a1-b0db-535f24a98089-kube-api-access-5pd54\") pod \"limitador-operator-controller-manager-859d7b78bd-zg57h\" (UID: \"f56ae7ab-5ad3-47a1-b0db-535f24a98089\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:40.971837 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:40.971752 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pd54\" (UniqueName: \"kubernetes.io/projected/f56ae7ab-5ad3-47a1-b0db-535f24a98089-kube-api-access-5pd54\") pod \"limitador-operator-controller-manager-859d7b78bd-zg57h\" (UID: \"f56ae7ab-5ad3-47a1-b0db-535f24a98089\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:41.014286 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:41.014259 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:41.167265 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:41.167238 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h"] Jul 23 18:58:41.169680 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:41.169652 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56ae7ab_5ad3_47a1_b0db_535f24a98089.slice/crio-cdc4b2c9f16d258aa5c9dcc77cb47315f04fcdcc0e844a8cf82621969913ae36 WatchSource:0}: Error finding container cdc4b2c9f16d258aa5c9dcc77cb47315f04fcdcc0e844a8cf82621969913ae36: Status 404 returned error can't find the container with id cdc4b2c9f16d258aa5c9dcc77cb47315f04fcdcc0e844a8cf82621969913ae36 Jul 23 18:58:41.684861 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:41.684802 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:41.815023 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:41.814988 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:42.163296 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:42.163254 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" event={"ID":"f56ae7ab-5ad3-47a1-b0db-535f24a98089","Type":"ContainerStarted","Data":"cdc4b2c9f16d258aa5c9dcc77cb47315f04fcdcc0e844a8cf82621969913ae36"} Jul 23 18:58:44.175285 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:44.175248 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" event={"ID":"f56ae7ab-5ad3-47a1-b0db-535f24a98089","Type":"ContainerStarted","Data":"abdbd397cdd6e4bd55100dbc16c31b617ddcdade30da874331cd7675219aea96"} Jul 23 18:58:44.175678 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:44.175321 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:44.193328 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:44.193283 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" podStartSLOduration=2.122830576 podStartE2EDuration="4.193269505s" totalImagesPullingTime="2.070438929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:58:44.192521629 +0000 UTC m=+476.773665847" watchObservedRunningTime="2026-07-23 18:58:44.193269505 +0000 UTC m=+476.774413725" Jul 23 18:58:45.017282 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.017246 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:45.017540 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.017519 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:45.021977 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.021957 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:45.189357 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.189333 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bf64c7869-vnw4l" Jul 23 18:58:45.257770 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.257741 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77dd4b66db-5f48g"] Jul 23 18:58:45.388492 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.388440 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:45.492958 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.492929 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:45.655690 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.655609 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:45.852645 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:45.852615 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:46.048594 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.048521 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k"] Jul 23 18:58:46.052037 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.052016 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.055629 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.055612 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jrjzl\"" Jul 23 18:58:46.072744 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.072720 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k"] Jul 23 18:58:46.077314 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.077282 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:46.183883 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.183846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c81b09e-ea3a-4199-b2b4-34a9484e1873-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.184068 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.183897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88hfd\" (UniqueName: \"kubernetes.io/projected/3c81b09e-ea3a-4199-b2b4-34a9484e1873-kube-api-access-88hfd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.285001 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.284968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c81b09e-ea3a-4199-b2b4-34a9484e1873-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.285416 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.285145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88hfd\" (UniqueName: \"kubernetes.io/projected/3c81b09e-ea3a-4199-b2b4-34a9484e1873-kube-api-access-88hfd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.285416 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.285347 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c81b09e-ea3a-4199-b2b4-34a9484e1873-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.294422 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.294398 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88hfd\" (UniqueName: \"kubernetes.io/projected/3c81b09e-ea3a-4199-b2b4-34a9484e1873-kube-api-access-88hfd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.362415 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.362383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:46.467121 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.467093 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:46.527711 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:46.527684 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k"] Jul 23 18:58:46.529234 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:46.529204 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c81b09e_ea3a_4199_b2b4_34a9484e1873.slice/crio-a75d276c8fa8dba05a78d9f024c220cf43f143fd5868de9ab115cf0d86eaae96 WatchSource:0}: Error finding container a75d276c8fa8dba05a78d9f024c220cf43f143fd5868de9ab115cf0d86eaae96: Status 404 returned error can't find the container with id a75d276c8fa8dba05a78d9f024c220cf43f143fd5868de9ab115cf0d86eaae96 Jul 23 18:58:47.200653 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:47.200615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" event={"ID":"3c81b09e-ea3a-4199-b2b4-34a9484e1873","Type":"ContainerStarted","Data":"a75d276c8fa8dba05a78d9f024c220cf43f143fd5868de9ab115cf0d86eaae96"} Jul 23 18:58:47.920096 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:47.920064 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:48.254661 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.254592 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7"] Jul 23 18:58:48.257869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.257854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:58:48.260452 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.260431 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ng7ct\"" Jul 23 18:58:48.271639 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.271611 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7"] Jul 23 18:58:48.417369 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.417328 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dknm\" (UniqueName: \"kubernetes.io/projected/81dec075-ac02-4c4b-a0c5-d54ad79127e8-kube-api-access-4dknm\") pod \"authorino-operator-75cc8d5cb7-cz4d7\" (UID: \"81dec075-ac02-4c4b-a0c5-d54ad79127e8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:58:48.518467 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.518390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dknm\" (UniqueName: \"kubernetes.io/projected/81dec075-ac02-4c4b-a0c5-d54ad79127e8-kube-api-access-4dknm\") pod \"authorino-operator-75cc8d5cb7-cz4d7\" (UID: \"81dec075-ac02-4c4b-a0c5-d54ad79127e8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:58:48.530410 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.530378 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dknm\" (UniqueName: \"kubernetes.io/projected/81dec075-ac02-4c4b-a0c5-d54ad79127e8-kube-api-access-4dknm\") pod \"authorino-operator-75cc8d5cb7-cz4d7\" (UID: \"81dec075-ac02-4c4b-a0c5-d54ad79127e8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:58:48.569385 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.569360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:58:48.727257 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:48.727230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7"] Jul 23 18:58:48.728910 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:58:48.728887 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81dec075_ac02_4c4b_a0c5_d54ad79127e8.slice/crio-148dd8d3b4725366ac34cb645150aceed6463faee73a5cee97a03b61c86873bf WatchSource:0}: Error finding container 148dd8d3b4725366ac34cb645150aceed6463faee73a5cee97a03b61c86873bf: Status 404 returned error can't find the container with id 148dd8d3b4725366ac34cb645150aceed6463faee73a5cee97a03b61c86873bf Jul 23 18:58:49.213722 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:49.213685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" event={"ID":"81dec075-ac02-4c4b-a0c5-d54ad79127e8","Type":"ContainerStarted","Data":"148dd8d3b4725366ac34cb645150aceed6463faee73a5cee97a03b61c86873bf"} Jul 23 18:58:49.479918 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:49.479833 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:50.524214 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:50.524178 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:50.663853 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:50.663792 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:52.234419 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:52.234373 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" event={"ID":"81dec075-ac02-4c4b-a0c5-d54ad79127e8","Type":"ContainerStarted","Data":"1b0498ceb7fc418fc59316595d3235dc9834238372d001e34d9c5ca16bc3a8a4"} Jul 23 18:58:52.235044 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:52.234472 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:58:52.266368 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:52.266256 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" podStartSLOduration=1.805508318 podStartE2EDuration="4.266234395s" totalImagesPullingTime="2.460726077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:58:52.264853231 +0000 UTC m=+484.845997451" watchObservedRunningTime="2026-07-23 18:58:52.266234395 +0000 UTC m=+484.847378616" Jul 23 18:58:52.309339 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:52.309292 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:53.241142 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:53.241100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" event={"ID":"3c81b09e-ea3a-4199-b2b4-34a9484e1873","Type":"ContainerStarted","Data":"95322b54a4cce4704a7d2a710675c3aef929e98a16224d52baacc3bc914eb063"} Jul 23 18:58:53.241535 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:53.241389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:58:53.272405 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:53.272351 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" podStartSLOduration=1.13209178 podStartE2EDuration="7.272337232s" totalImagesPullingTime="6.140245452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:58:53.270935181 +0000 UTC m=+485.852079401" watchObservedRunningTime="2026-07-23 18:58:53.272337232 +0000 UTC m=+485.853481451" Jul 23 18:58:53.657927 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:53.657887 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:53.761326 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:53.761274 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:55.188412 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:55.188373 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:58:56.354874 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:56.354795 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:56.460143 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:56.460113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:56.888208 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:56.888173 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:56.987079 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:56.987046 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:58:58.382026 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:58:58.381996 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:03.243807 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:03.243777 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cz4d7" Jul 23 18:59:04.248874 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:04.248843 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:05.849800 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:05.849766 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:05.997811 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:05.997782 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:06.104407 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.104307 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:06.136250 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.136213 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k"] Jul 23 18:59:06.136519 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.136479 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" containerName="manager" containerID="cri-o://95322b54a4cce4704a7d2a710675c3aef929e98a16224d52baacc3bc914eb063" gracePeriod=2 Jul 23 18:59:06.151287 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.151254 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k"] Jul 23 18:59:06.169548 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.169512 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h"] Jul 23 18:59:06.169858 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.169777 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" containerName="manager" containerID="cri-o://abdbd397cdd6e4bd55100dbc16c31b617ddcdade30da874331cd7675219aea96" gracePeriod=2 Jul 23 18:59:06.171902 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.171882 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm"] Jul 23 18:59:06.172281 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.172269 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" containerName="manager" Jul 23 18:59:06.172326 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.172284 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" containerName="manager" Jul 23 18:59:06.172357 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.172347 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" containerName="manager" Jul 23 18:59:06.175800 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.175779 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.185382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.185349 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k"] Jul 23 18:59:06.193540 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.193379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.193698 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.193264 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h"] Jul 23 18:59:06.196119 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.196006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm"] Jul 23 18:59:06.205878 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.205580 2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f56ae7ab-5ad3-47a1-b0db-535f24a98089\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-zg57h\": pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:06.209546 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.209525 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k"] Jul 23 18:59:06.210017 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.209972 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:06.240712 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.240677 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:06.242655 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.242625 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:06.271001 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.270970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1240a700-4d57-4026-881d-211a53648086-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qstm\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.271127 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.271010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/630bfeb3-d0c8-48f0-bb40-eb2f45baca04-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2s25k\" (UID: \"630bfeb3-d0c8-48f0-bb40-eb2f45baca04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.271127 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.271065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pswkp\" (UniqueName: \"kubernetes.io/projected/630bfeb3-d0c8-48f0-bb40-eb2f45baca04-kube-api-access-pswkp\") pod \"kuadrant-operator-controller-manager-86878c769f-2s25k\" (UID: \"630bfeb3-d0c8-48f0-bb40-eb2f45baca04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.271127 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.271104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87cwd\" (UniqueName: \"kubernetes.io/projected/1240a700-4d57-4026-881d-211a53648086-kube-api-access-87cwd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qstm\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.313558 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.313528 2578 generic.go:356] "Generic (PLEG): container finished" podID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" containerID="abdbd397cdd6e4bd55100dbc16c31b617ddcdade30da874331cd7675219aea96" exitCode=0 Jul 23 18:59:06.315308 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.315289 2578 generic.go:356] "Generic (PLEG): container finished" podID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" containerID="95322b54a4cce4704a7d2a710675c3aef929e98a16224d52baacc3bc914eb063" exitCode=0 Jul 23 18:59:06.372650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.372562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1240a700-4d57-4026-881d-211a53648086-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qstm\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.372650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.372620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/630bfeb3-d0c8-48f0-bb40-eb2f45baca04-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2s25k\" (UID: \"630bfeb3-d0c8-48f0-bb40-eb2f45baca04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.372868 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.372671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pswkp\" (UniqueName: \"kubernetes.io/projected/630bfeb3-d0c8-48f0-bb40-eb2f45baca04-kube-api-access-pswkp\") pod \"kuadrant-operator-controller-manager-86878c769f-2s25k\" (UID: \"630bfeb3-d0c8-48f0-bb40-eb2f45baca04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.372868 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.372720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cwd\" (UniqueName: \"kubernetes.io/projected/1240a700-4d57-4026-881d-211a53648086-kube-api-access-87cwd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qstm\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.373059 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.373034 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/630bfeb3-d0c8-48f0-bb40-eb2f45baca04-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2s25k\" (UID: \"630bfeb3-d0c8-48f0-bb40-eb2f45baca04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.373147 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.373122 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1240a700-4d57-4026-881d-211a53648086-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qstm\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.387036 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.387012 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pswkp\" (UniqueName: \"kubernetes.io/projected/630bfeb3-d0c8-48f0-bb40-eb2f45baca04-kube-api-access-pswkp\") pod \"kuadrant-operator-controller-manager-86878c769f-2s25k\" (UID: \"630bfeb3-d0c8-48f0-bb40-eb2f45baca04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.387186 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.387147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cwd\" (UniqueName: \"kubernetes.io/projected/1240a700-4d57-4026-881d-211a53648086-kube-api-access-87cwd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qstm\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.417376 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.417357 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:06.419438 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.419416 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:06.419834 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.419794 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:06.422055 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.422032 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:06.424280 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.424261 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:06.426175 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:06.426157 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:06.473491 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.473464 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88hfd\" (UniqueName: \"kubernetes.io/projected/3c81b09e-ea3a-4199-b2b4-34a9484e1873-kube-api-access-88hfd\") pod \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " Jul 23 18:59:06.473599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.473525 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c81b09e-ea3a-4199-b2b4-34a9484e1873-extensions-socket-volume\") pod \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\" (UID: \"3c81b09e-ea3a-4199-b2b4-34a9484e1873\") " Jul 23 18:59:06.473599 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.473559 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pd54\" (UniqueName: \"kubernetes.io/projected/f56ae7ab-5ad3-47a1-b0db-535f24a98089-kube-api-access-5pd54\") pod \"f56ae7ab-5ad3-47a1-b0db-535f24a98089\" (UID: \"f56ae7ab-5ad3-47a1-b0db-535f24a98089\") " Jul 23 18:59:06.474114 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.474059 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c81b09e-ea3a-4199-b2b4-34a9484e1873-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3c81b09e-ea3a-4199-b2b4-34a9484e1873" (UID: "3c81b09e-ea3a-4199-b2b4-34a9484e1873"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:59:06.475680 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.475660 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56ae7ab-5ad3-47a1-b0db-535f24a98089-kube-api-access-5pd54" (OuterVolumeSpecName: "kube-api-access-5pd54") pod "f56ae7ab-5ad3-47a1-b0db-535f24a98089" (UID: "f56ae7ab-5ad3-47a1-b0db-535f24a98089"). InnerVolumeSpecName "kube-api-access-5pd54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:59:06.475680 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.475668 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c81b09e-ea3a-4199-b2b4-34a9484e1873-kube-api-access-88hfd" (OuterVolumeSpecName: "kube-api-access-88hfd") pod "3c81b09e-ea3a-4199-b2b4-34a9484e1873" (UID: "3c81b09e-ea3a-4199-b2b4-34a9484e1873"). InnerVolumeSpecName "kube-api-access-88hfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:59:06.563597 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.563542 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:06.569625 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.569608 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:06.574751 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.574724 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c81b09e-ea3a-4199-b2b4-34a9484e1873-extensions-socket-volume\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:06.574869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.574750 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pd54\" (UniqueName: \"kubernetes.io/projected/f56ae7ab-5ad3-47a1-b0db-535f24a98089-kube-api-access-5pd54\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:06.574869 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.574768 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88hfd\" (UniqueName: \"kubernetes.io/projected/3c81b09e-ea3a-4199-b2b4-34a9484e1873-kube-api-access-88hfd\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:06.728729 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.728698 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm"] Jul 23 18:59:06.731624 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:59:06.731602 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1240a700_4d57_4026_881d_211a53648086.slice/crio-3c1ac49cbe72b7e86ac29dacabb3accdf645250fd1d4f0cfae0f32e3bfe9a9f0 WatchSource:0}: Error finding container 3c1ac49cbe72b7e86ac29dacabb3accdf645250fd1d4f0cfae0f32e3bfe9a9f0: Status 404 returned error can't find the container with id 3c1ac49cbe72b7e86ac29dacabb3accdf645250fd1d4f0cfae0f32e3bfe9a9f0 Jul 23 18:59:06.742428 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:06.742408 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k"] Jul 23 18:59:06.744616 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:59:06.744598 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630bfeb3_d0c8_48f0_bb40_eb2f45baca04.slice/crio-991dc0c73331ea8a2729c36d6d54cb9a321351970c28dc5bb897f590d1276db9 WatchSource:0}: Error finding container 991dc0c73331ea8a2729c36d6d54cb9a321351970c28dc5bb897f590d1276db9: Status 404 returned error can't find the container with id 991dc0c73331ea8a2729c36d6d54cb9a321351970c28dc5bb897f590d1276db9 Jul 23 18:59:07.322959 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.322923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" event={"ID":"630bfeb3-d0c8-48f0-bb40-eb2f45baca04","Type":"ContainerStarted","Data":"21cd80fd1f30b8619ae9780294b3f16e70e5ca5c93c474652bfb1fef33aaca05"} Jul 23 18:59:07.323378 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.322975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" event={"ID":"630bfeb3-d0c8-48f0-bb40-eb2f45baca04","Type":"ContainerStarted","Data":"991dc0c73331ea8a2729c36d6d54cb9a321351970c28dc5bb897f590d1276db9"} Jul 23 18:59:07.323378 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.323019 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:07.324356 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.324337 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:07.324479 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.324379 2578 scope.go:117] "RemoveContainer" containerID="abdbd397cdd6e4bd55100dbc16c31b617ddcdade30da874331cd7675219aea96" Jul 23 18:59:07.325591 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:07.325496 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:07.326276 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.326255 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" event={"ID":"1240a700-4d57-4026-881d-211a53648086","Type":"ContainerStarted","Data":"c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08"} Jul 23 18:59:07.326367 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.326296 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" event={"ID":"1240a700-4d57-4026-881d-211a53648086","Type":"ContainerStarted","Data":"3c1ac49cbe72b7e86ac29dacabb3accdf645250fd1d4f0cfae0f32e3bfe9a9f0"} Jul 23 18:59:07.326367 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.326347 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:07.327518 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:07.327497 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:07.328016 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.328000 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:07.333650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.333635 2578 scope.go:117] "RemoveContainer" containerID="95322b54a4cce4704a7d2a710675c3aef929e98a16224d52baacc3bc914eb063" Jul 23 18:59:07.363887 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.363848 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" podStartSLOduration=1.363834287 podStartE2EDuration="1.363834287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:59:07.361586672 +0000 UTC m=+499.942730924" watchObservedRunningTime="2026-07-23 18:59:07.363834287 +0000 UTC m=+499.944978498" Jul 23 18:59:07.383024 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.382982 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" podStartSLOduration=1.382968125 podStartE2EDuration="1.382968125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:59:07.381957067 +0000 UTC m=+499.963101299" watchObservedRunningTime="2026-07-23 18:59:07.382968125 +0000 UTC m=+499.964112345" Jul 23 18:59:07.384047 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:07.384017 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:07.385994 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:07.385970 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:07.927739 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:07.927699 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zg57h\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zg57h" Jul 23 18:59:07.930094 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:07.930059 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tfm5k\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tfm5k" Jul 23 18:59:07.930284 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.930260 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c81b09e-ea3a-4199-b2b4-34a9484e1873" path="/var/lib/kubelet/pods/3c81b09e-ea3a-4199-b2b4-34a9484e1873/volumes" Jul 23 18:59:07.930661 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:07.930642 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" path="/var/lib/kubelet/pods/f56ae7ab-5ad3-47a1-b0db-535f24a98089/volumes" Jul 23 18:59:08.122771 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:08.122731 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:08.206352 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:08.206262 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:10.283121 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.283064 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77dd4b66db-5f48g" podUID="22daede8-67cb-4ef7-9ffb-79854a109eb7" containerName="console" containerID="cri-o://a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca" gracePeriod=15 Jul 23 18:59:10.530842 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.530796 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:59:10.609200 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609160 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-serving-cert\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.609200 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609213 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-config\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.609412 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609265 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf2bb\" (UniqueName: \"kubernetes.io/projected/22daede8-67cb-4ef7-9ffb-79854a109eb7-kube-api-access-wf2bb\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.609412 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609302 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-service-ca\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.609412 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609338 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-oauth-serving-cert\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.609412 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-trusted-ca-bundle\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.610030 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-oauth-config\") pod \"22daede8-67cb-4ef7-9ffb-79854a109eb7\" (UID: \"22daede8-67cb-4ef7-9ffb-79854a109eb7\") " Jul 23 18:59:10.610030 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609799 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-config" (OuterVolumeSpecName: "console-config") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:59:10.610030 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609853 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:59:10.610030 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609919 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-service-ca" (OuterVolumeSpecName: "service-ca") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:59:10.610030 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.609868 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:59:10.611851 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.611801 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:59:10.611987 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.611960 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22daede8-67cb-4ef7-9ffb-79854a109eb7-kube-api-access-wf2bb" (OuterVolumeSpecName: "kube-api-access-wf2bb") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "kube-api-access-wf2bb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:59:10.612080 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.612062 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "22daede8-67cb-4ef7-9ffb-79854a109eb7" (UID: "22daede8-67cb-4ef7-9ffb-79854a109eb7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:59:10.710769 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710739 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:10.710769 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710765 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-oauth-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:10.710769 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710776 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-trusted-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:10.710990 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710785 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-oauth-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:10.710990 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710794 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:10.710990 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710801 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22daede8-67cb-4ef7-9ffb-79854a109eb7-console-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:10.710990 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:10.710809 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf2bb\" (UniqueName: \"kubernetes.io/projected/22daede8-67cb-4ef7-9ffb-79854a109eb7-kube-api-access-wf2bb\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:11.353725 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.353684 2578 generic.go:356] "Generic (PLEG): container finished" podID="22daede8-67cb-4ef7-9ffb-79854a109eb7" containerID="a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca" exitCode=0 Jul 23 18:59:11.353725 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.353731 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dd4b66db-5f48g" Jul 23 18:59:11.354236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.353747 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dd4b66db-5f48g" event={"ID":"22daede8-67cb-4ef7-9ffb-79854a109eb7","Type":"ContainerDied","Data":"a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca"} Jul 23 18:59:11.354236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.353796 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dd4b66db-5f48g" event={"ID":"22daede8-67cb-4ef7-9ffb-79854a109eb7","Type":"ContainerDied","Data":"f3a27887175681c8acca0011f0ed21e93da7cb81f54a0714427af23f6083f27c"} Jul 23 18:59:11.354236 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.353844 2578 scope.go:117] "RemoveContainer" containerID="a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca" Jul 23 18:59:11.363501 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.363481 2578 scope.go:117] "RemoveContainer" containerID="a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca" Jul 23 18:59:11.363810 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:11.363782 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca\": container with ID starting with a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca not found: ID does not exist" containerID="a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca" Jul 23 18:59:11.363906 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.363845 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca"} err="failed to get container status \"a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca\": rpc error: code = NotFound desc = could not find container \"a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca\": container with ID starting with a8a68e77e744fde070e9925457306a80bfd24c6b3a04bc0e2ce7bf4a636f9dca not found: ID does not exist" Jul 23 18:59:11.382513 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.382489 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77dd4b66db-5f48g"] Jul 23 18:59:11.384697 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.384678 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77dd4b66db-5f48g"] Jul 23 18:59:11.922437 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:11.922403 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22daede8-67cb-4ef7-9ffb-79854a109eb7" path="/var/lib/kubelet/pods/22daede8-67cb-4ef7-9ffb-79854a109eb7/volumes" Jul 23 18:59:12.476382 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:12.476352 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:12.576306 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:12.576279 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:14.927685 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:14.927652 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:15.039227 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:15.039195 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:18.338985 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.338951 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2s25k" Jul 23 18:59:18.339422 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.339040 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:18.516637 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.516600 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm"] Jul 23 18:59:18.516816 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.516787 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" podUID="1240a700-4d57-4026-881d-211a53648086" containerName="manager" containerID="cri-o://c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08" gracePeriod=10 Jul 23 18:59:18.772451 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.772423 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:18.885418 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.885381 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1240a700-4d57-4026-881d-211a53648086-extensions-socket-volume\") pod \"1240a700-4d57-4026-881d-211a53648086\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " Jul 23 18:59:18.885708 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.885494 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87cwd\" (UniqueName: \"kubernetes.io/projected/1240a700-4d57-4026-881d-211a53648086-kube-api-access-87cwd\") pod \"1240a700-4d57-4026-881d-211a53648086\" (UID: \"1240a700-4d57-4026-881d-211a53648086\") " Jul 23 18:59:18.885797 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.885746 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1240a700-4d57-4026-881d-211a53648086-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1240a700-4d57-4026-881d-211a53648086" (UID: "1240a700-4d57-4026-881d-211a53648086"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:59:18.887725 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.887702 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1240a700-4d57-4026-881d-211a53648086-kube-api-access-87cwd" (OuterVolumeSpecName: "kube-api-access-87cwd") pod "1240a700-4d57-4026-881d-211a53648086" (UID: "1240a700-4d57-4026-881d-211a53648086"). InnerVolumeSpecName "kube-api-access-87cwd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:59:18.986351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.986320 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1240a700-4d57-4026-881d-211a53648086-extensions-socket-volume\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:18.986351 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:18.986342 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87cwd\" (UniqueName: \"kubernetes.io/projected/1240a700-4d57-4026-881d-211a53648086-kube-api-access-87cwd\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:19.398891 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.398854 2578 generic.go:356] "Generic (PLEG): container finished" podID="1240a700-4d57-4026-881d-211a53648086" containerID="c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08" exitCode=0 Jul 23 18:59:19.399322 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.398943 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" Jul 23 18:59:19.399322 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.398988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" event={"ID":"1240a700-4d57-4026-881d-211a53648086","Type":"ContainerDied","Data":"c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08"} Jul 23 18:59:19.399322 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.399034 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm" event={"ID":"1240a700-4d57-4026-881d-211a53648086","Type":"ContainerDied","Data":"3c1ac49cbe72b7e86ac29dacabb3accdf645250fd1d4f0cfae0f32e3bfe9a9f0"} Jul 23 18:59:19.399322 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.399060 2578 scope.go:117] "RemoveContainer" containerID="c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08" Jul 23 18:59:19.408546 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.408527 2578 scope.go:117] "RemoveContainer" containerID="c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08" Jul 23 18:59:19.408798 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:19.408781 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08\": container with ID starting with c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08 not found: ID does not exist" containerID="c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08" Jul 23 18:59:19.408897 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.408803 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08"} err="failed to get container status \"c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08\": rpc error: code = NotFound desc = could not find container \"c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08\": container with ID starting with c57aac5aee72c1a5370b5b60e83afe87f33d5fc804508509391b4f6ac8e87a08 not found: ID does not exist" Jul 23 18:59:19.430082 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.430051 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm"] Jul 23 18:59:19.433390 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.433368 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qstm"] Jul 23 18:59:19.922579 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:19.922537 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1240a700-4d57-4026-881d-211a53648086" path="/var/lib/kubelet/pods/1240a700-4d57-4026-881d-211a53648086/volumes" Jul 23 18:59:23.795205 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:23.795171 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:23.891575 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:23.891534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:26.219206 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:26.219171 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:26.320087 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:26.320055 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:27.825401 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:27.825364 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:30.674284 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:30.674247 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:30.778032 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:30.777995 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:33.586754 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:33.586692 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:33.686795 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:33.686755 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:34.908320 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908290 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6"] Jul 23 18:59:34.908684 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908654 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" containerName="manager" Jul 23 18:59:34.908684 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908664 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" containerName="manager" Jul 23 18:59:34.908684 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908676 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1240a700-4d57-4026-881d-211a53648086" containerName="manager" Jul 23 18:59:34.908684 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908681 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1240a700-4d57-4026-881d-211a53648086" containerName="manager" Jul 23 18:59:34.908834 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908688 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22daede8-67cb-4ef7-9ffb-79854a109eb7" containerName="console" Jul 23 18:59:34.908834 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908694 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22daede8-67cb-4ef7-9ffb-79854a109eb7" containerName="console" Jul 23 18:59:34.908834 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908749 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56ae7ab-5ad3-47a1-b0db-535f24a98089" containerName="manager" Jul 23 18:59:34.908834 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908758 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22daede8-67cb-4ef7-9ffb-79854a109eb7" containerName="console" Jul 23 18:59:34.908834 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.908764 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1240a700-4d57-4026-881d-211a53648086" containerName="manager" Jul 23 18:59:34.913526 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.913505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:34.916572 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.916553 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pkvfl\"" Jul 23 18:59:34.932946 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.932916 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6"] Jul 23 18:59:34.942323 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:34.942294 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:35.020772 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.020731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.020772 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.020775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021037 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.020805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021037 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.020851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021037 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.020880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021037 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.020942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021186 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.021057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021186 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.021114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.021283 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.021258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wmdm\" (UniqueName: \"kubernetes.io/projected/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-kube-api-access-6wmdm\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.121773 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.121729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.121982 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.121789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wmdm\" (UniqueName: \"kubernetes.io/projected/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-kube-api-access-6wmdm\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.121982 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.121887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.121982 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.121918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.121982 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.121951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.121982 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.121980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122256 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122256 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122256 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122256 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122210 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122422 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122306 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122422 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122410 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122531 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122482 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.122666 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.122644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.124515 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.124488 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.124653 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.124636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.134774 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.134750 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wmdm\" (UniqueName: \"kubernetes.io/projected/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-kube-api-access-6wmdm\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.136017 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.135992 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9d5bf26-fb59-4dfd-9181-1ad7611412e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-js5r6\" (UID: \"c9d5bf26-fb59-4dfd-9181-1ad7611412e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.141502 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.141481 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:35.223562 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.223483 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:35.369643 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.369617 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6"] Jul 23 18:59:35.371050 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:59:35.371014 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d5bf26_fb59_4dfd_9181_1ad7611412e1.slice/crio-914040195aa14d98cda08cd3be9a35191ec54c1cf0fd4db94897d2549c79fd25 WatchSource:0}: Error finding container 914040195aa14d98cda08cd3be9a35191ec54c1cf0fd4db94897d2549c79fd25: Status 404 returned error can't find the container with id 914040195aa14d98cda08cd3be9a35191ec54c1cf0fd4db94897d2549c79fd25 Jul 23 18:59:35.373047 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.373018 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:59:35.373151 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.373087 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:59:35.373151 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.373134 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 18:59:35.485912 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.485802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" event={"ID":"c9d5bf26-fb59-4dfd-9181-1ad7611412e1","Type":"ContainerStarted","Data":"bda32b4fce9aef0cbf196310fdd90789ad384b690b9f9cd1184cadf9def740ea"} Jul 23 18:59:35.485912 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.485880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" event={"ID":"c9d5bf26-fb59-4dfd-9181-1ad7611412e1","Type":"ContainerStarted","Data":"914040195aa14d98cda08cd3be9a35191ec54c1cf0fd4db94897d2549c79fd25"} Jul 23 18:59:35.512766 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:35.512715 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" podStartSLOduration=1.512697441 podStartE2EDuration="1.512697441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:59:35.512038129 +0000 UTC m=+528.093182348" watchObservedRunningTime="2026-07-23 18:59:35.512697441 +0000 UTC m=+528.093841661" Jul 23 18:59:36.224640 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:36.224603 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:36.229491 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:36.229458 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:36.492232 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:36.492150 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:36.493145 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:36.493123 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 18:59:36.773850 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:36.773743 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:36.882841 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:36.882794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:37.491140 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.491102 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xj55l"] Jul 23 18:59:37.493639 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.493615 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:37.496212 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.496192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xhvgb\"" Jul 23 18:59:37.506635 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.506613 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xj55l"] Jul 23 18:59:37.527914 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.527881 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:37.631518 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.631479 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:37.638249 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.638222 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s42c"] Jul 23 18:59:37.640679 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.640660 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 18:59:37.646170 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.646146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2mkb\" (UniqueName: \"kubernetes.io/projected/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7-kube-api-access-f2mkb\") pod \"authorino-58cd955675-xj55l\" (UID: \"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7\") " pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:37.650156 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.650134 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s42c"] Jul 23 18:59:37.747645 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.747559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2mkb\" (UniqueName: \"kubernetes.io/projected/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7-kube-api-access-f2mkb\") pod \"authorino-58cd955675-xj55l\" (UID: \"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7\") " pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:37.747645 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.747621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfz5x\" (UniqueName: \"kubernetes.io/projected/abe49caa-b170-458d-b04a-fe34dcb18394-kube-api-access-rfz5x\") pod \"authorino-75d556bd77-9s42c\" (UID: \"abe49caa-b170-458d-b04a-fe34dcb18394\") " pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 18:59:37.758203 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.758177 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2mkb\" (UniqueName: \"kubernetes.io/projected/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7-kube-api-access-f2mkb\") pod \"authorino-58cd955675-xj55l\" (UID: \"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7\") " pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:37.804130 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.804094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:37.848099 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.848064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfz5x\" (UniqueName: \"kubernetes.io/projected/abe49caa-b170-458d-b04a-fe34dcb18394-kube-api-access-rfz5x\") pod \"authorino-75d556bd77-9s42c\" (UID: \"abe49caa-b170-458d-b04a-fe34dcb18394\") " pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 18:59:37.860965 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.860874 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfz5x\" (UniqueName: \"kubernetes.io/projected/abe49caa-b170-458d-b04a-fe34dcb18394-kube-api-access-rfz5x\") pod \"authorino-75d556bd77-9s42c\" (UID: \"abe49caa-b170-458d-b04a-fe34dcb18394\") " pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 18:59:37.947772 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.947746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xj55l"] Jul 23 18:59:37.949407 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:59:37.949381 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7179d3e_f6d6_4f5a_aebc_2a58eb0627c7.slice/crio-f5412bad49fdd1cf4b170adb2825c0256cf4ca10bb224c78bb7beaadf1d831ca WatchSource:0}: Error finding container f5412bad49fdd1cf4b170adb2825c0256cf4ca10bb224c78bb7beaadf1d831ca: Status 404 returned error can't find the container with id f5412bad49fdd1cf4b170adb2825c0256cf4ca10bb224c78bb7beaadf1d831ca Jul 23 18:59:37.950111 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:37.950092 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 18:59:38.090500 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:38.090409 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s42c"] Jul 23 18:59:38.093110 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:59:38.093072 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabe49caa_b170_458d_b04a_fe34dcb18394.slice/crio-a5a37024085a8089539f6ddacd339ad16f84ec74a39af864faeab525caf5a568 WatchSource:0}: Error finding container a5a37024085a8089539f6ddacd339ad16f84ec74a39af864faeab525caf5a568: Status 404 returned error can't find the container with id a5a37024085a8089539f6ddacd339ad16f84ec74a39af864faeab525caf5a568 Jul 23 18:59:38.504375 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:38.504337 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s42c" event={"ID":"abe49caa-b170-458d-b04a-fe34dcb18394","Type":"ContainerStarted","Data":"a5a37024085a8089539f6ddacd339ad16f84ec74a39af864faeab525caf5a568"} Jul 23 18:59:38.505469 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:38.505446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xj55l" event={"ID":"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7","Type":"ContainerStarted","Data":"f5412bad49fdd1cf4b170adb2825c0256cf4ca10bb224c78bb7beaadf1d831ca"} Jul 23 18:59:38.985709 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:38.985671 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:39.263107 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.263023 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 18:59:39.266130 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.266105 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.269359 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.269336 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kzh8g\"" Jul 23 18:59:39.269682 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.269660 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 18:59:39.280892 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.280851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 18:59:39.296772 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.296714 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 18:59:39.374646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.374295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gfg9\" (UniqueName: \"kubernetes.io/projected/87d5541a-73b1-47a9-933e-475187b83350-kube-api-access-2gfg9\") pod \"limitador-limitador-54d6d988d5-w9z4b\" (UID: \"87d5541a-73b1-47a9-933e-475187b83350\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.374646 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.374432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87d5541a-73b1-47a9-933e-475187b83350-config-file\") pod \"limitador-limitador-54d6d988d5-w9z4b\" (UID: \"87d5541a-73b1-47a9-933e-475187b83350\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.475364 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.475327 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87d5541a-73b1-47a9-933e-475187b83350-config-file\") pod \"limitador-limitador-54d6d988d5-w9z4b\" (UID: \"87d5541a-73b1-47a9-933e-475187b83350\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.475577 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.475407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gfg9\" (UniqueName: \"kubernetes.io/projected/87d5541a-73b1-47a9-933e-475187b83350-kube-api-access-2gfg9\") pod \"limitador-limitador-54d6d988d5-w9z4b\" (UID: \"87d5541a-73b1-47a9-933e-475187b83350\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.476801 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.476501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87d5541a-73b1-47a9-933e-475187b83350-config-file\") pod \"limitador-limitador-54d6d988d5-w9z4b\" (UID: \"87d5541a-73b1-47a9-933e-475187b83350\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.507744 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.507687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gfg9\" (UniqueName: \"kubernetes.io/projected/87d5541a-73b1-47a9-933e-475187b83350-kube-api-access-2gfg9\") pod \"limitador-limitador-54d6d988d5-w9z4b\" (UID: \"87d5541a-73b1-47a9-933e-475187b83350\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.584006 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.583969 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:39.793500 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:39.793452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 18:59:39.809697 ip-10-0-141-105 kubenswrapper[2578]: W0723 18:59:39.809650 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d5541a_73b1_47a9_933e_475187b83350.slice/crio-a87efa7216b135a162c307884102c1dd982186992075d8d28b8f86bcb126a485 WatchSource:0}: Error finding container a87efa7216b135a162c307884102c1dd982186992075d8d28b8f86bcb126a485: Status 404 returned error can't find the container with id a87efa7216b135a162c307884102c1dd982186992075d8d28b8f86bcb126a485 Jul 23 18:59:40.525804 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:40.525764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" event={"ID":"87d5541a-73b1-47a9-933e-475187b83350","Type":"ContainerStarted","Data":"a87efa7216b135a162c307884102c1dd982186992075d8d28b8f86bcb126a485"} Jul 23 18:59:42.540593 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.540555 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s42c" event={"ID":"abe49caa-b170-458d-b04a-fe34dcb18394","Type":"ContainerStarted","Data":"1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec"} Jul 23 18:59:42.542145 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.542121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xj55l" event={"ID":"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7","Type":"ContainerStarted","Data":"720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0"} Jul 23 18:59:42.560043 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.559995 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9s42c" podStartSLOduration=1.8882967910000001 podStartE2EDuration="5.559980017s" totalImagesPullingTime="3.671683226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:59:42.557000744 +0000 UTC m=+535.138144968" watchObservedRunningTime="2026-07-23 18:59:42.559980017 +0000 UTC m=+535.141124236" Jul 23 18:59:42.577960 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.577911 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xj55l" podStartSLOduration=1.7473572499999999 podStartE2EDuration="5.577892282s" totalImagesPullingTime="3.830535032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:59:42.575164817 +0000 UTC m=+535.156309037" watchObservedRunningTime="2026-07-23 18:59:42.577892282 +0000 UTC m=+535.159036503" Jul 23 18:59:42.605650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.605617 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xj55l"] Jul 23 18:59:42.792948 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.792811 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:42.895029 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:42.894997 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:43.549583 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:43.549541 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" event={"ID":"87d5541a-73b1-47a9-933e-475187b83350","Type":"ContainerStarted","Data":"4264330453fa6f0659f3e855699f1cdc7723bd2a1787a4eaddd2992f6ed64acd"} Jul 23 18:59:43.550058 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:43.549736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:43.568594 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:43.568547 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" podStartSLOduration=1.050826994 podStartE2EDuration="4.568529848s" totalImagesPullingTime="3.517702854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:59:43.567631887 +0000 UTC m=+536.148776118" watchObservedRunningTime="2026-07-23 18:59:43.568529848 +0000 UTC m=+536.149674069" Jul 23 18:59:44.555439 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:44.555394 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xj55l" podUID="e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" containerName="authorino" containerID="cri-o://720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0" gracePeriod=30 Jul 23 18:59:44.816645 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:44.816585 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:44.823872 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:44.823852 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2mkb\" (UniqueName: \"kubernetes.io/projected/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7-kube-api-access-f2mkb\") pod \"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7\" (UID: \"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7\") " Jul 23 18:59:44.825942 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:44.825912 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7-kube-api-access-f2mkb" (OuterVolumeSpecName: "kube-api-access-f2mkb") pod "e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" (UID: "e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7"). InnerVolumeSpecName "kube-api-access-f2mkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:59:44.924701 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:44.924669 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2mkb\" (UniqueName: \"kubernetes.io/projected/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7-kube-api-access-f2mkb\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 18:59:45.249142 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.249112 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:45.349550 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.349491 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:45.562014 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.561932 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" containerID="720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0" exitCode=0 Jul 23 18:59:45.562014 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.561991 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xj55l" Jul 23 18:59:45.562458 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.562009 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xj55l" event={"ID":"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7","Type":"ContainerDied","Data":"720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0"} Jul 23 18:59:45.562458 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.562052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xj55l" event={"ID":"e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7","Type":"ContainerDied","Data":"f5412bad49fdd1cf4b170adb2825c0256cf4ca10bb224c78bb7beaadf1d831ca"} Jul 23 18:59:45.562458 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.562070 2578 scope.go:117] "RemoveContainer" containerID="720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0" Jul 23 18:59:45.571650 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.571627 2578 scope.go:117] "RemoveContainer" containerID="720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0" Jul 23 18:59:45.571915 ip-10-0-141-105 kubenswrapper[2578]: E0723 18:59:45.571896 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0\": container with ID starting with 720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0 not found: ID does not exist" containerID="720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0" Jul 23 18:59:45.571986 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.571925 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0"} err="failed to get container status \"720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0\": rpc error: code = NotFound desc = could not find container \"720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0\": container with ID starting with 720421b39d3917ed7fb35c411b1e9ba6680656373e49c6574373ce5a809b5ea0 not found: ID does not exist" Jul 23 18:59:45.585616 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.585591 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xj55l"] Jul 23 18:59:45.592301 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.592279 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xj55l"] Jul 23 18:59:45.923400 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:45.923367 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" path="/var/lib/kubelet/pods/e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7/volumes" Jul 23 18:59:54.556607 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:54.556578 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-w9z4b" Jul 23 18:59:57.841201 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:57.841144 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:57.931531 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:57.931497 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 18:59:58.329587 ip-10-0-141-105 kubenswrapper[2578]: I0723 18:59:58.329562 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:10.789578 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:10.789546 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:11.015535 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.015503 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-b7hrt"] Jul 23 19:00:11.015962 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.015948 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" containerName="authorino" Jul 23 19:00:11.016016 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.015964 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" containerName="authorino" Jul 23 19:00:11.016077 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.016067 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7179d3e-f6d6-4f5a-aebc-2a58eb0627c7" containerName="authorino" Jul 23 19:00:11.066251 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.066179 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-b7hrt"] Jul 23 19:00:11.066392 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.066273 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:11.100236 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.100194 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:11.161849 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.161791 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-757rw\" (UniqueName: \"kubernetes.io/projected/1736d978-907e-46ff-97d8-0f985965db08-kube-api-access-757rw\") pod \"authorino-5d4b67c446-b7hrt\" (UID: \"1736d978-907e-46ff-97d8-0f985965db08\") " pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:11.248856 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.248806 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-b7hrt"] Jul 23 19:00:11.249159 ip-10-0-141-105 kubenswrapper[2578]: E0723 19:00:11.249136 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-757rw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-b7hrt" podUID="1736d978-907e-46ff-97d8-0f985965db08" Jul 23 19:00:11.262841 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.262780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-757rw\" (UniqueName: \"kubernetes.io/projected/1736d978-907e-46ff-97d8-0f985965db08-kube-api-access-757rw\") pod \"authorino-5d4b67c446-b7hrt\" (UID: \"1736d978-907e-46ff-97d8-0f985965db08\") " pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:11.274179 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.274150 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-757rw\" (UniqueName: \"kubernetes.io/projected/1736d978-907e-46ff-97d8-0f985965db08-kube-api-access-757rw\") pod \"authorino-5d4b67c446-b7hrt\" (UID: \"1736d978-907e-46ff-97d8-0f985965db08\") " pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:11.281250 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.281226 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66647c5c4f-wpbl5"] Jul 23 19:00:11.303653 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.303624 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66647c5c4f-wpbl5"] Jul 23 19:00:11.303837 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.303731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:11.363541 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.363509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krl7g\" (UniqueName: \"kubernetes.io/projected/5718131e-a15d-4525-a997-6378c9f47e68-kube-api-access-krl7g\") pod \"authorino-66647c5c4f-wpbl5\" (UID: \"5718131e-a15d-4525-a997-6378c9f47e68\") " pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:11.464273 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.464231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krl7g\" (UniqueName: \"kubernetes.io/projected/5718131e-a15d-4525-a997-6378c9f47e68-kube-api-access-krl7g\") pod \"authorino-66647c5c4f-wpbl5\" (UID: \"5718131e-a15d-4525-a997-6378c9f47e68\") " pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:11.466448 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.466426 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66647c5c4f-wpbl5"] Jul 23 19:00:11.466678 ip-10-0-141-105 kubenswrapper[2578]: E0723 19:00:11.466658 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-krl7g], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-66647c5c4f-wpbl5" podUID="5718131e-a15d-4525-a997-6378c9f47e68" Jul 23 19:00:11.476215 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.476192 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krl7g\" (UniqueName: \"kubernetes.io/projected/5718131e-a15d-4525-a997-6378c9f47e68-kube-api-access-krl7g\") pod \"authorino-66647c5c4f-wpbl5\" (UID: \"5718131e-a15d-4525-a997-6378c9f47e68\") " pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:11.493634 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.493608 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7ccff6f7bf-ql6z6"] Jul 23 19:00:11.516221 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.516197 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ccff6f7bf-ql6z6"] Jul 23 19:00:11.516350 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.516285 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.518985 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.518964 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 19:00:11.666604 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.666513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44562fa5-b57b-4506-9745-5173be1f23af-tls-cert\") pod \"authorino-7ccff6f7bf-ql6z6\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.666604 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.666581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqhnh\" (UniqueName: \"kubernetes.io/projected/44562fa5-b57b-4506-9745-5173be1f23af-kube-api-access-hqhnh\") pod \"authorino-7ccff6f7bf-ql6z6\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.704243 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.704207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:11.704408 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.704214 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:11.729076 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.729045 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:11.745267 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.745239 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:11.767669 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.767640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqhnh\" (UniqueName: \"kubernetes.io/projected/44562fa5-b57b-4506-9745-5173be1f23af-kube-api-access-hqhnh\") pod \"authorino-7ccff6f7bf-ql6z6\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.767846 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.767699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44562fa5-b57b-4506-9745-5173be1f23af-tls-cert\") pod \"authorino-7ccff6f7bf-ql6z6\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.770308 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.770285 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44562fa5-b57b-4506-9745-5173be1f23af-tls-cert\") pod \"authorino-7ccff6f7bf-ql6z6\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.777922 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.777901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqhnh\" (UniqueName: \"kubernetes.io/projected/44562fa5-b57b-4506-9745-5173be1f23af-kube-api-access-hqhnh\") pod \"authorino-7ccff6f7bf-ql6z6\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.826813 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.826779 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:00:11.868528 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.868500 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krl7g\" (UniqueName: \"kubernetes.io/projected/5718131e-a15d-4525-a997-6378c9f47e68-kube-api-access-krl7g\") pod \"5718131e-a15d-4525-a997-6378c9f47e68\" (UID: \"5718131e-a15d-4525-a997-6378c9f47e68\") " Jul 23 19:00:11.868678 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.868543 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-757rw\" (UniqueName: \"kubernetes.io/projected/1736d978-907e-46ff-97d8-0f985965db08-kube-api-access-757rw\") pod \"1736d978-907e-46ff-97d8-0f985965db08\" (UID: \"1736d978-907e-46ff-97d8-0f985965db08\") " Jul 23 19:00:11.870742 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.870712 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5718131e-a15d-4525-a997-6378c9f47e68-kube-api-access-krl7g" (OuterVolumeSpecName: "kube-api-access-krl7g") pod "5718131e-a15d-4525-a997-6378c9f47e68" (UID: "5718131e-a15d-4525-a997-6378c9f47e68"). InnerVolumeSpecName "kube-api-access-krl7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:00:11.870855 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.870784 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1736d978-907e-46ff-97d8-0f985965db08-kube-api-access-757rw" (OuterVolumeSpecName: "kube-api-access-757rw") pod "1736d978-907e-46ff-97d8-0f985965db08" (UID: "1736d978-907e-46ff-97d8-0f985965db08"). InnerVolumeSpecName "kube-api-access-757rw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:00:11.970134 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.970107 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krl7g\" (UniqueName: \"kubernetes.io/projected/5718131e-a15d-4525-a997-6378c9f47e68-kube-api-access-krl7g\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 19:00:11.970134 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.970134 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-757rw\" (UniqueName: \"kubernetes.io/projected/1736d978-907e-46ff-97d8-0f985965db08-kube-api-access-757rw\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 19:00:11.987594 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:11.987561 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ccff6f7bf-ql6z6"] Jul 23 19:00:11.991949 ip-10-0-141-105 kubenswrapper[2578]: W0723 19:00:11.991914 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44562fa5_b57b_4506_9745_5173be1f23af.slice/crio-d82d352da801dbe1070bcaaf30462775216c392ba580817618f57409bba5cac8 WatchSource:0}: Error finding container d82d352da801dbe1070bcaaf30462775216c392ba580817618f57409bba5cac8: Status 404 returned error can't find the container with id d82d352da801dbe1070bcaaf30462775216c392ba580817618f57409bba5cac8 Jul 23 19:00:12.719089 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.719051 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" event={"ID":"44562fa5-b57b-4506-9745-5173be1f23af","Type":"ContainerStarted","Data":"cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007"} Jul 23 19:00:12.719249 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.719101 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" event={"ID":"44562fa5-b57b-4506-9745-5173be1f23af","Type":"ContainerStarted","Data":"d82d352da801dbe1070bcaaf30462775216c392ba580817618f57409bba5cac8"} Jul 23 19:00:12.719249 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.719184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66647c5c4f-wpbl5" Jul 23 19:00:12.719378 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.719361 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-b7hrt" Jul 23 19:00:12.738524 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.738467 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" podStartSLOduration=1.111937456 podStartE2EDuration="1.73845229s" totalImagesPullingTime="626.514834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:00:12.735990779 +0000 UTC m=+565.317135010" watchObservedRunningTime="2026-07-23 19:00:12.73845229 +0000 UTC m=+565.319596535" Jul 23 19:00:12.775679 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.775614 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s42c"] Jul 23 19:00:12.775887 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.775841 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9s42c" podUID="abe49caa-b170-458d-b04a-fe34dcb18394" containerName="authorino" containerID="cri-o://1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec" gracePeriod=30 Jul 23 19:00:12.779609 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.779549 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66647c5c4f-wpbl5"] Jul 23 19:00:12.783152 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.783126 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66647c5c4f-wpbl5"] Jul 23 19:00:12.812153 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.812122 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-b7hrt"] Jul 23 19:00:12.815910 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:12.815883 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-b7hrt"] Jul 23 19:00:13.021807 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.021784 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 19:00:13.179866 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.179754 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfz5x\" (UniqueName: \"kubernetes.io/projected/abe49caa-b170-458d-b04a-fe34dcb18394-kube-api-access-rfz5x\") pod \"abe49caa-b170-458d-b04a-fe34dcb18394\" (UID: \"abe49caa-b170-458d-b04a-fe34dcb18394\") " Jul 23 19:00:13.181896 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.181873 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe49caa-b170-458d-b04a-fe34dcb18394-kube-api-access-rfz5x" (OuterVolumeSpecName: "kube-api-access-rfz5x") pod "abe49caa-b170-458d-b04a-fe34dcb18394" (UID: "abe49caa-b170-458d-b04a-fe34dcb18394"). InnerVolumeSpecName "kube-api-access-rfz5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:00:13.281392 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.281365 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfz5x\" (UniqueName: \"kubernetes.io/projected/abe49caa-b170-458d-b04a-fe34dcb18394-kube-api-access-rfz5x\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 19:00:13.726069 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.726036 2578 generic.go:356] "Generic (PLEG): container finished" podID="abe49caa-b170-458d-b04a-fe34dcb18394" containerID="1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec" exitCode=0 Jul 23 19:00:13.726273 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.726075 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s42c" Jul 23 19:00:13.726273 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.726119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s42c" event={"ID":"abe49caa-b170-458d-b04a-fe34dcb18394","Type":"ContainerDied","Data":"1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec"} Jul 23 19:00:13.726273 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.726161 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s42c" event={"ID":"abe49caa-b170-458d-b04a-fe34dcb18394","Type":"ContainerDied","Data":"a5a37024085a8089539f6ddacd339ad16f84ec74a39af864faeab525caf5a568"} Jul 23 19:00:13.726273 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.726180 2578 scope.go:117] "RemoveContainer" containerID="1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec" Jul 23 19:00:13.735570 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.735553 2578 scope.go:117] "RemoveContainer" containerID="1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec" Jul 23 19:00:13.735808 ip-10-0-141-105 kubenswrapper[2578]: E0723 19:00:13.735788 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec\": container with ID starting with 1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec not found: ID does not exist" containerID="1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec" Jul 23 19:00:13.735887 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.735833 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec"} err="failed to get container status \"1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec\": rpc error: code = NotFound desc = could not find container \"1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec\": container with ID starting with 1f66b466bd9d71cd9c7a5114b2cb0d8515ff2137a4392ddd1eee4c035301caec not found: ID does not exist" Jul 23 19:00:13.750489 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.750459 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s42c"] Jul 23 19:00:13.752677 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.752653 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s42c"] Jul 23 19:00:13.923247 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.923216 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1736d978-907e-46ff-97d8-0f985965db08" path="/var/lib/kubelet/pods/1736d978-907e-46ff-97d8-0f985965db08/volumes" Jul 23 19:00:13.923432 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.923420 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5718131e-a15d-4525-a997-6378c9f47e68" path="/var/lib/kubelet/pods/5718131e-a15d-4525-a997-6378c9f47e68/volumes" Jul 23 19:00:13.923618 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:13.923607 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abe49caa-b170-458d-b04a-fe34dcb18394" path="/var/lib/kubelet/pods/abe49caa-b170-458d-b04a-fe34dcb18394/volumes" Jul 23 19:00:17.501764 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:17.501727 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:17.572880 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:17.572848 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:17.642050 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:17.642009 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:17.734100 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:17.734052 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:20.835729 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:20.835692 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:20.935068 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:20.935027 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:32.572585 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:32.572526 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:32.672972 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:32.672939 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:39.708996 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:39.708962 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:53.017256 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:53.017213 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:54.631142 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:54.631105 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:00:54.719603 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:00:54.719560 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:01:02.987251 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:02.987216 2578 generic.go:356] "Generic (PLEG): container finished" podID="c9d5bf26-fb59-4dfd-9181-1ad7611412e1" containerID="bda32b4fce9aef0cbf196310fdd90789ad384b690b9f9cd1184cadf9def740ea" exitCode=0 Jul 23 19:01:02.987774 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:02.987291 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" event={"ID":"c9d5bf26-fb59-4dfd-9181-1ad7611412e1","Type":"ContainerDied","Data":"bda32b4fce9aef0cbf196310fdd90789ad384b690b9f9cd1184cadf9def740ea"} Jul 23 19:01:02.987870 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:02.987813 2578 scope.go:117] "RemoveContainer" containerID="bda32b4fce9aef0cbf196310fdd90789ad384b690b9f9cd1184cadf9def740ea" Jul 23 19:01:02.988892 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:02.988867 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 19:01:02.989008 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:02.988931 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 19:01:02.989008 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:02.988970 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 19:01:03.999112 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:03.999065 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" event={"ID":"c9d5bf26-fb59-4dfd-9181-1ad7611412e1","Type":"ContainerStarted","Data":"0dad9b04708f4c7fe3e190803d32dae1c7876a90f2e4956e410920fef8064575"} Jul 23 19:01:04.224305 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:04.224262 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 19:01:04.225505 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:04.225483 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-js5r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 23 19:01:04.225618 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:04.225541 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" podUID="c9d5bf26-fb59-4dfd-9181-1ad7611412e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 23 19:01:04.706782 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:04.706750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:05.035061 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:05.034969 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:05.224536 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:05.224500 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-js5r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 23 19:01:05.224709 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:05.224566 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" podUID="c9d5bf26-fb59-4dfd-9181-1ad7611412e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 23 19:01:06.224260 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:06.224218 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-js5r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 23 19:01:06.224608 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:06.224284 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" podUID="c9d5bf26-fb59-4dfd-9181-1ad7611412e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 23 19:01:06.492489 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:06.492383 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 19:01:07.224075 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:07.224041 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-js5r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 23 19:01:07.224268 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:07.224100 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" podUID="c9d5bf26-fb59-4dfd-9181-1ad7611412e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 23 19:01:08.228216 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:08.228177 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 19:01:08.229179 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:08.229163 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-js5r6" Jul 23 19:01:19.647293 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:19.647262 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:32.277599 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:32.277555 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:32.743762 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:32.743726 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:01:42.456529 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:42.456492 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:43.155144 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:43.155107 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:01:49.642044 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.642005 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6425k"] Jul 23 19:01:49.642424 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.642411 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe49caa-b170-458d-b04a-fe34dcb18394" containerName="authorino" Jul 23 19:01:49.642468 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.642426 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe49caa-b170-458d-b04a-fe34dcb18394" containerName="authorino" Jul 23 19:01:49.642509 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.642499 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe49caa-b170-458d-b04a-fe34dcb18394" containerName="authorino" Jul 23 19:01:49.644807 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.644788 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.647732 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.647708 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nclw6\"" Jul 23 19:01:49.649110 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.649084 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 19:01:49.649246 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.649144 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 19:01:49.649246 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.649155 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 19:01:49.659264 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.659243 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6425k"] Jul 23 19:01:49.717753 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.717718 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:49.734018 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.733980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.734018 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.734021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.734196 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.734059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jc7t\" (UniqueName: \"kubernetes.io/projected/16e857e1-0737-4076-828a-b0dd4b77d9df-kube-api-access-4jc7t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.734196 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.734141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.734196 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.734158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e857e1-0737-4076-828a-b0dd4b77d9df-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.734294 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.734210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835477 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835477 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e857e1-0737-4076-828a-b0dd4b77d9df-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835735 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835735 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835735 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835735 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jc7t\" (UniqueName: \"kubernetes.io/projected/16e857e1-0737-4076-828a-b0dd4b77d9df-kube-api-access-4jc7t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835999 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.835999 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.835958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.836078 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.836001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.837973 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.837951 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16e857e1-0737-4076-828a-b0dd4b77d9df-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.838105 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.838089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e857e1-0737-4076-828a-b0dd4b77d9df-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.844073 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.844051 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jc7t\" (UniqueName: \"kubernetes.io/projected/16e857e1-0737-4076-828a-b0dd4b77d9df-kube-api-access-4jc7t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6425k\" (UID: \"16e857e1-0737-4076-828a-b0dd4b77d9df\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:49.955014 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:49.954914 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:50.102898 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:50.102871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6425k"] Jul 23 19:01:50.105013 ip-10-0-141-105 kubenswrapper[2578]: W0723 19:01:50.104982 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e857e1_0737_4076_828a_b0dd4b77d9df.slice/crio-e6fd4714cd568ae991fe370875a763d74eecc1783614b95fcf8ac9b541d21bb1 WatchSource:0}: Error finding container e6fd4714cd568ae991fe370875a763d74eecc1783614b95fcf8ac9b541d21bb1: Status 404 returned error can't find the container with id e6fd4714cd568ae991fe370875a763d74eecc1783614b95fcf8ac9b541d21bb1 Jul 23 19:01:50.106355 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:50.106334 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 19:01:50.246226 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:50.246134 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" event={"ID":"16e857e1-0737-4076-828a-b0dd4b77d9df","Type":"ContainerStarted","Data":"e6fd4714cd568ae991fe370875a763d74eecc1783614b95fcf8ac9b541d21bb1"} Jul 23 19:01:50.957686 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:50.957648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:01:51.253968 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:51.253935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" event={"ID":"16e857e1-0737-4076-828a-b0dd4b77d9df","Type":"ContainerStarted","Data":"cf564f6008ddc01411f15f521e5269d97f0f9d9e27843ea8df30dfb271c3f4f5"} Jul 23 19:01:51.274082 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:51.274009 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" podStartSLOduration=1.2355763 podStartE2EDuration="2.273987717s" totalImagesPullingTime="1.038411417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:51.273149675 +0000 UTC m=+663.854293896" watchObservedRunningTime="2026-07-23 19:01:51.273987717 +0000 UTC m=+663.855131938" Jul 23 19:01:58.170038 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:58.169992 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:59.687408 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:59.687371 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:01:59.955917 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:59.955798 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:59.955917 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:59.955895 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:01:59.968901 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:01:59.968875 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:02:00.316810 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:00.316727 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6425k" Jul 23 19:02:15.215700 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:15.215652 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:18.059584 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:18.059542 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:02:24.657716 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:24.657680 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:28.349371 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:28.349332 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w9z4b"] Jul 23 19:02:32.130650 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:32.130598 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:43.930952 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:43.930848 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:44.029734 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:02:44.029700 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:31.402376 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.402337 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57dd8c56d-qzpzp"] Jul 23 19:03:31.405008 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.404991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.415180 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.415158 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57dd8c56d-qzpzp"] Jul 23 19:03:31.524445 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.524415 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:31.556144 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.556108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/14be5701-46ea-4ff6-b8a6-5fd525be56df-tls-cert\") pod \"authorino-57dd8c56d-qzpzp\" (UID: \"14be5701-46ea-4ff6-b8a6-5fd525be56df\") " pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.556302 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.556233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qt6n\" (UniqueName: \"kubernetes.io/projected/14be5701-46ea-4ff6-b8a6-5fd525be56df-kube-api-access-4qt6n\") pod \"authorino-57dd8c56d-qzpzp\" (UID: \"14be5701-46ea-4ff6-b8a6-5fd525be56df\") " pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.624285 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.624246 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:31.657473 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.657391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qt6n\" (UniqueName: \"kubernetes.io/projected/14be5701-46ea-4ff6-b8a6-5fd525be56df-kube-api-access-4qt6n\") pod \"authorino-57dd8c56d-qzpzp\" (UID: \"14be5701-46ea-4ff6-b8a6-5fd525be56df\") " pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.657473 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.657445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/14be5701-46ea-4ff6-b8a6-5fd525be56df-tls-cert\") pod \"authorino-57dd8c56d-qzpzp\" (UID: \"14be5701-46ea-4ff6-b8a6-5fd525be56df\") " pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.659956 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.659937 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/14be5701-46ea-4ff6-b8a6-5fd525be56df-tls-cert\") pod \"authorino-57dd8c56d-qzpzp\" (UID: \"14be5701-46ea-4ff6-b8a6-5fd525be56df\") " pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.669806 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.669783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qt6n\" (UniqueName: \"kubernetes.io/projected/14be5701-46ea-4ff6-b8a6-5fd525be56df-kube-api-access-4qt6n\") pod \"authorino-57dd8c56d-qzpzp\" (UID: \"14be5701-46ea-4ff6-b8a6-5fd525be56df\") " pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.714663 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.714625 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57dd8c56d-qzpzp" Jul 23 19:03:31.858039 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:31.858014 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57dd8c56d-qzpzp"] Jul 23 19:03:31.860259 ip-10-0-141-105 kubenswrapper[2578]: W0723 19:03:31.860234 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14be5701_46ea_4ff6_b8a6_5fd525be56df.slice/crio-e0615e98e567511f4145aae6020987390c57839575ce1bf5183623bfeb9e616f WatchSource:0}: Error finding container e0615e98e567511f4145aae6020987390c57839575ce1bf5183623bfeb9e616f: Status 404 returned error can't find the container with id e0615e98e567511f4145aae6020987390c57839575ce1bf5183623bfeb9e616f Jul 23 19:03:32.677011 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.676927 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:32.779471 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.779435 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:32.804271 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.804237 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57dd8c56d-qzpzp" event={"ID":"14be5701-46ea-4ff6-b8a6-5fd525be56df","Type":"ContainerStarted","Data":"f1b63584b5af5baafb35a940992328f951a430ff7f750f941614cd799b152f27"} Jul 23 19:03:32.804443 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.804287 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57dd8c56d-qzpzp" event={"ID":"14be5701-46ea-4ff6-b8a6-5fd525be56df","Type":"ContainerStarted","Data":"e0615e98e567511f4145aae6020987390c57839575ce1bf5183623bfeb9e616f"} Jul 23 19:03:32.872240 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.872173 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57dd8c56d-qzpzp" podStartSLOduration=1.322733235 podStartE2EDuration="1.872150441s" totalImagesPullingTime="549.417206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:03:32.867984915 +0000 UTC m=+765.449129137" watchObservedRunningTime="2026-07-23 19:03:32.872150441 +0000 UTC m=+765.453294662" Jul 23 19:03:32.946430 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.946350 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7ccff6f7bf-ql6z6"] Jul 23 19:03:32.946574 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:32.946533 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" podUID="44562fa5-b57b-4506-9745-5173be1f23af" containerName="authorino" containerID="cri-o://cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007" gracePeriod=30 Jul 23 19:03:33.216753 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.216730 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:03:33.375090 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.374274 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44562fa5-b57b-4506-9745-5173be1f23af-tls-cert\") pod \"44562fa5-b57b-4506-9745-5173be1f23af\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " Jul 23 19:03:33.375090 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.374356 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqhnh\" (UniqueName: \"kubernetes.io/projected/44562fa5-b57b-4506-9745-5173be1f23af-kube-api-access-hqhnh\") pod \"44562fa5-b57b-4506-9745-5173be1f23af\" (UID: \"44562fa5-b57b-4506-9745-5173be1f23af\") " Jul 23 19:03:33.378152 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.378099 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44562fa5-b57b-4506-9745-5173be1f23af-kube-api-access-hqhnh" (OuterVolumeSpecName: "kube-api-access-hqhnh") pod "44562fa5-b57b-4506-9745-5173be1f23af" (UID: "44562fa5-b57b-4506-9745-5173be1f23af"). InnerVolumeSpecName "kube-api-access-hqhnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:03:33.394531 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.394494 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44562fa5-b57b-4506-9745-5173be1f23af-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "44562fa5-b57b-4506-9745-5173be1f23af" (UID: "44562fa5-b57b-4506-9745-5173be1f23af"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 19:03:33.475626 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.475531 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqhnh\" (UniqueName: \"kubernetes.io/projected/44562fa5-b57b-4506-9745-5173be1f23af-kube-api-access-hqhnh\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 19:03:33.475626 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.475569 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44562fa5-b57b-4506-9745-5173be1f23af-tls-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\"" Jul 23 19:03:33.814782 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.814642 2578 generic.go:356] "Generic (PLEG): container finished" podID="44562fa5-b57b-4506-9745-5173be1f23af" containerID="cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007" exitCode=0 Jul 23 19:03:33.814782 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.814705 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" Jul 23 19:03:33.815289 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.814853 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" event={"ID":"44562fa5-b57b-4506-9745-5173be1f23af","Type":"ContainerDied","Data":"cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007"} Jul 23 19:03:33.815289 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.814898 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ccff6f7bf-ql6z6" event={"ID":"44562fa5-b57b-4506-9745-5173be1f23af","Type":"ContainerDied","Data":"d82d352da801dbe1070bcaaf30462775216c392ba580817618f57409bba5cac8"} Jul 23 19:03:33.815289 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.814923 2578 scope.go:117] "RemoveContainer" containerID="cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007" Jul 23 19:03:33.827244 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.826435 2578 scope.go:117] "RemoveContainer" containerID="cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007" Jul 23 19:03:33.827244 ip-10-0-141-105 kubenswrapper[2578]: E0723 19:03:33.826918 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007\": container with ID starting with cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007 not found: ID does not exist" containerID="cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007" Jul 23 19:03:33.827244 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.826953 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007"} err="failed to get container status \"cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007\": rpc error: code = NotFound desc = could not find container \"cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007\": container with ID starting with cb781bac021bed9a90ca814212f2cb0dbd9b70d374986c5b452153ed5f3da007 not found: ID does not exist" Jul 23 19:03:33.849374 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.849338 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7ccff6f7bf-ql6z6"] Jul 23 19:03:33.861320 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.861289 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7ccff6f7bf-ql6z6"] Jul 23 19:03:33.923229 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:33.923194 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44562fa5-b57b-4506-9745-5173be1f23af" path="/var/lib/kubelet/pods/44562fa5-b57b-4506-9745-5173be1f23af/volumes" Jul 23 19:03:52.270901 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:52.270863 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57dd8c56d-qzpzp_14be5701-46ea-4ff6-b8a6-5fd525be56df/authorino/0.log" Jul 23 19:03:57.520843 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:57.520794 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dtwnv_3dcf84a9-0697-4547-a309-db90a958dca6/manager/0.log" Jul 23 19:03:57.632173 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:57.632138 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jrfwz_a644b371-42b3-4fb0-a9c8-4ff7c2f71762/manager/0.log" Jul 23 19:03:57.902874 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:57.902843 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-29h4l_2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6/server/0.log" Jul 23 19:03:58.019274 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:58.019240 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nqrct_01252b47-6b6d-44ee-95c7-57d8e0fd1615/manager/0.log" Jul 23 19:03:58.413594 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:58.413561 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c495ff4d9-lpxjj_5e0089e0-fc30-4c90-9a19-2978a6883436/manager/0.log" Jul 23 19:03:58.421247 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:58.421228 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c495ff4d9-lpxjj_5e0089e0-fc30-4c90-9a19-2978a6883436/copy-manifests/0.log" Jul 23 19:03:58.584316 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:58.584279 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:58.669156 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:58.669051 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:59.637393 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.637364 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld_e7718953-8cc3-4cba-acef-ec0b1d1fd4b7/util/0.log" Jul 23 19:03:59.644787 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.644764 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld_e7718953-8cc3-4cba-acef-ec0b1d1fd4b7/pull/0.log" Jul 23 19:03:59.653046 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.653029 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld_e7718953-8cc3-4cba-acef-ec0b1d1fd4b7/extract/0.log" Jul 23 19:03:59.761586 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.761555 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz_658de733-506e-41ff-9acb-ca3b818424c8/util/0.log" Jul 23 19:03:59.768796 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.768773 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz_658de733-506e-41ff-9acb-ca3b818424c8/pull/0.log" Jul 23 19:03:59.775149 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.775131 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz_658de733-506e-41ff-9acb-ca3b818424c8/extract/0.log" Jul 23 19:03:59.898368 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.898289 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp_7eca061d-f05a-4597-a3c4-387e3ef3b1e3/util/0.log" Jul 23 19:03:59.906779 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.906760 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp_7eca061d-f05a-4597-a3c4-387e3ef3b1e3/pull/0.log" Jul 23 19:03:59.916301 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:03:59.916283 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp_7eca061d-f05a-4597-a3c4-387e3ef3b1e3/extract/0.log" Jul 23 19:04:00.030916 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.030888 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk_c746bf6c-3ec6-4501-b7ba-5a54c4f75598/util/0.log" Jul 23 19:04:00.038852 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.038810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk_c746bf6c-3ec6-4501-b7ba-5a54c4f75598/pull/0.log" Jul 23 19:04:00.049609 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.049588 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk_c746bf6c-3ec6-4501-b7ba-5a54c4f75598/extract/0.log" Jul 23 19:04:00.164292 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.164208 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57dd8c56d-qzpzp_14be5701-46ea-4ff6-b8a6-5fd525be56df/authorino/0.log" Jul 23 19:04:00.281939 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.281908 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cz4d7_81dec075-ac02-4c4b-a0c5-d54ad79127e8/manager/0.log" Jul 23 19:04:00.638048 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.638024 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vz8fd_77beb20c-6335-4565-ae02-e4976d548945/registry-server/0.log" Jul 23 19:04:00.763572 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.763543 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2s25k_630bfeb3-d0c8-48f0-bb40-eb2f45baca04/manager/0.log" Jul 23 19:04:00.918700 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:00.918619 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-w9z4b_87d5541a-73b1-47a9-933e-475187b83350/limitador/0.log" Jul 23 19:04:01.380610 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:01.380581 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48svgj_027ae2f8-e98c-4668-870a-90d35ce919ed/istio-proxy/0.log" Jul 23 19:04:01.616352 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:01.616321 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8f784977d-7fbz7_75dc15d6-0580-4360-97f0-a049f8851c2e/kube-auth-proxy/0.log" Jul 23 19:04:01.852440 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:01.852407 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/1.log" Jul 23 19:04:02.770945 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:02.770917 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-6425k_16e857e1-0737-4076-828a-b0dd4b77d9df/main/0.log" Jul 23 19:04:08.593223 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.593142 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ckwzf/must-gather-sfwjt"] Jul 23 19:04:08.593662 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.593603 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44562fa5-b57b-4506-9745-5173be1f23af" containerName="authorino" Jul 23 19:04:08.593662 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.593621 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44562fa5-b57b-4506-9745-5173be1f23af" containerName="authorino" Jul 23 19:04:08.593769 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.593716 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44562fa5-b57b-4506-9745-5173be1f23af" containerName="authorino" Jul 23 19:04:08.598051 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.598029 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.600366 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.600346 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ckwzf\"/\"default-dockercfg-kfdr8\"" Jul 23 19:04:08.600715 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.600693 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ckwzf\"/\"kube-root-ca.crt\"" Jul 23 19:04:08.600979 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.600964 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ckwzf\"/\"openshift-service-ca.crt\"" Jul 23 19:04:08.606378 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.606356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckwzf/must-gather-sfwjt"] Jul 23 19:04:08.612078 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.612060 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:08.695430 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.695401 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:08.696553 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.696535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab3a6626-4788-4301-a0b7-c3a1c7513e03-must-gather-output\") pod \"must-gather-sfwjt\" (UID: \"ab3a6626-4788-4301-a0b7-c3a1c7513e03\") " pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.696607 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.696568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9dv\" (UniqueName: \"kubernetes.io/projected/ab3a6626-4788-4301-a0b7-c3a1c7513e03-kube-api-access-rb9dv\") pod \"must-gather-sfwjt\" (UID: \"ab3a6626-4788-4301-a0b7-c3a1c7513e03\") " pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.797909 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.797877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab3a6626-4788-4301-a0b7-c3a1c7513e03-must-gather-output\") pod \"must-gather-sfwjt\" (UID: \"ab3a6626-4788-4301-a0b7-c3a1c7513e03\") " pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.798063 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.797915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9dv\" (UniqueName: \"kubernetes.io/projected/ab3a6626-4788-4301-a0b7-c3a1c7513e03-kube-api-access-rb9dv\") pod \"must-gather-sfwjt\" (UID: \"ab3a6626-4788-4301-a0b7-c3a1c7513e03\") " pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.798216 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.798196 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab3a6626-4788-4301-a0b7-c3a1c7513e03-must-gather-output\") pod \"must-gather-sfwjt\" (UID: \"ab3a6626-4788-4301-a0b7-c3a1c7513e03\") " pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.806720 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.806697 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9dv\" (UniqueName: \"kubernetes.io/projected/ab3a6626-4788-4301-a0b7-c3a1c7513e03-kube-api-access-rb9dv\") pod \"must-gather-sfwjt\" (UID: \"ab3a6626-4788-4301-a0b7-c3a1c7513e03\") " pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:08.907193 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:08.907116 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckwzf/must-gather-sfwjt" Jul 23 19:04:09.042401 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:09.042377 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckwzf/must-gather-sfwjt"] Jul 23 19:04:09.043700 ip-10-0-141-105 kubenswrapper[2578]: W0723 19:04:09.043668 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab3a6626_4788_4301_a0b7_c3a1c7513e03.slice/crio-64e1df29d7b2cf48cff0e7b54fd3babf7a0f88ec0ad1ed06b01e091d9475fcdc WatchSource:0}: Error finding container 64e1df29d7b2cf48cff0e7b54fd3babf7a0f88ec0ad1ed06b01e091d9475fcdc: Status 404 returned error can't find the container with id 64e1df29d7b2cf48cff0e7b54fd3babf7a0f88ec0ad1ed06b01e091d9475fcdc Jul 23 19:04:10.018469 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:10.018439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckwzf/must-gather-sfwjt" event={"ID":"ab3a6626-4788-4301-a0b7-c3a1c7513e03","Type":"ContainerStarted","Data":"f93495ad79e82949265a19366126eb1e937e64a03776a6d2514f62133e624ea2"} Jul 23 19:04:10.018962 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:10.018931 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckwzf/must-gather-sfwjt" event={"ID":"ab3a6626-4788-4301-a0b7-c3a1c7513e03","Type":"ContainerStarted","Data":"64e1df29d7b2cf48cff0e7b54fd3babf7a0f88ec0ad1ed06b01e091d9475fcdc"} Jul 23 19:04:11.031697 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:11.031660 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckwzf/must-gather-sfwjt" event={"ID":"ab3a6626-4788-4301-a0b7-c3a1c7513e03","Type":"ContainerStarted","Data":"94b0ca87cb931e6291e7a69fceb659491da32aaffb70c7807bb71dbf1848d731"} Jul 23 19:04:11.051329 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:11.051277 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ckwzf/must-gather-sfwjt" podStartSLOduration=2.183475398 podStartE2EDuration="3.051257066s" totalImagesPullingTime="867.781668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:04:11.049328452 +0000 UTC m=+803.630472672" watchObservedRunningTime="2026-07-23 19:04:11.051257066 +0000 UTC m=+803.632401286" Jul 23 19:04:11.960409 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:11.960383 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wphxg_a943505d-6390-4fe2-b647-70f281a4d58a/global-pull-secret-syncer/0.log" Jul 23 19:04:12.053074 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:12.053039 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4c2k5_d49df187-4039-4b20-b24f-9eb4e33d5620/konnectivity-agent/0.log" Jul 23 19:04:12.198371 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:12.198324 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-105.ec2.internal_3d37fd728dd468b7ee1c820042e75b2d/haproxy/0.log" Jul 23 19:04:15.590094 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.590006 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld_e7718953-8cc3-4cba-acef-ec0b1d1fd4b7/extract/0.log" Jul 23 19:04:15.623221 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.623166 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld_e7718953-8cc3-4cba-acef-ec0b1d1fd4b7/util/0.log" Jul 23 19:04:15.658865 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.658812 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qkld_e7718953-8cc3-4cba-acef-ec0b1d1fd4b7/pull/0.log" Jul 23 19:04:15.696097 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.696062 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz_658de733-506e-41ff-9acb-ca3b818424c8/extract/0.log" Jul 23 19:04:15.733445 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.733408 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz_658de733-506e-41ff-9acb-ca3b818424c8/util/0.log" Jul 23 19:04:15.774082 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.774022 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0887cz_658de733-506e-41ff-9acb-ca3b818424c8/pull/0.log" Jul 23 19:04:15.809504 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.809473 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp_7eca061d-f05a-4597-a3c4-387e3ef3b1e3/extract/0.log" Jul 23 19:04:15.837632 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.837600 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp_7eca061d-f05a-4597-a3c4-387e3ef3b1e3/util/0.log" Jul 23 19:04:15.866264 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.866178 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737x2tp_7eca061d-f05a-4597-a3c4-387e3ef3b1e3/pull/0.log" Jul 23 19:04:15.910675 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.910490 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk_c746bf6c-3ec6-4501-b7ba-5a54c4f75598/extract/0.log" Jul 23 19:04:15.950429 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.950326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk_c746bf6c-3ec6-4501-b7ba-5a54c4f75598/util/0.log" Jul 23 19:04:15.978306 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:15.978188 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jt2zk_c746bf6c-3ec6-4501-b7ba-5a54c4f75598/pull/0.log" Jul 23 19:04:16.034022 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:16.033926 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57dd8c56d-qzpzp_14be5701-46ea-4ff6-b8a6-5fd525be56df/authorino/0.log" Jul 23 19:04:16.078840 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:16.078745 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cz4d7_81dec075-ac02-4c4b-a0c5-d54ad79127e8/manager/0.log" Jul 23 19:04:16.220599 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:16.220463 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vz8fd_77beb20c-6335-4565-ae02-e4976d548945/registry-server/0.log" Jul 23 19:04:16.289027 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:16.288996 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2s25k_630bfeb3-d0c8-48f0-bb40-eb2f45baca04/manager/0.log" Jul 23 19:04:16.321866 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:16.321814 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-w9z4b_87d5541a-73b1-47a9-933e-475187b83350/limitador/0.log" Jul 23 19:04:18.510079 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.510052 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-hm777_f402f241-1c8f-4057-a3a4-212d5bf6bf5e/monitoring-plugin/0.log" Jul 23 19:04:18.739419 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.739369 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z2g9t_845b068d-c7f2-401f-9213-528263a4bee5/node-exporter/0.log" Jul 23 19:04:18.773231 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.773152 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z2g9t_845b068d-c7f2-401f-9213-528263a4bee5/kube-rbac-proxy/0.log" Jul 23 19:04:18.803282 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.803259 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z2g9t_845b068d-c7f2-401f-9213-528263a4bee5/init-textfile/0.log" Jul 23 19:04:18.850662 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.850637 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-fdmbq_49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4/kube-rbac-proxy-main/0.log" Jul 23 19:04:18.878310 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.878276 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-fdmbq_49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4/kube-rbac-proxy-self/0.log" Jul 23 19:04:18.914901 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:18.914867 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-fdmbq_49f2dc49-4fc1-42a9-8fa2-bb18d9ce3ab4/openshift-state-metrics/0.log" Jul 23 19:04:19.237472 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.237441 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-z5qxv_1ad01d67-ceac-45a6-b473-e54f34910705/prometheus-operator/0.log" Jul 23 19:04:19.271838 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.271782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-z5qxv_1ad01d67-ceac-45a6-b473-e54f34910705/kube-rbac-proxy/0.log" Jul 23 19:04:19.303231 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.303196 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-26rst_889397c4-85f2-41d2-8739-0f3591c12e03/prometheus-operator-admission-webhook/0.log" Jul 23 19:04:19.352038 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.352010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76f54b79fb-b5hj6_a622d335-309c-4f22-8cd1-ca53cef7972d/telemeter-client/0.log" Jul 23 19:04:19.378169 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.378128 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76f54b79fb-b5hj6_a622d335-309c-4f22-8cd1-ca53cef7972d/reload/0.log" Jul 23 19:04:19.410377 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.410354 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76f54b79fb-b5hj6_a622d335-309c-4f22-8cd1-ca53cef7972d/kube-rbac-proxy/0.log" Jul 23 19:04:19.449294 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.449264 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599ff9bf95-fjqv6_7866138c-9592-4ed3-89a5-df4363514b42/thanos-query/0.log" Jul 23 19:04:19.477010 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.476969 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599ff9bf95-fjqv6_7866138c-9592-4ed3-89a5-df4363514b42/kube-rbac-proxy-web/0.log" Jul 23 19:04:19.505253 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.505226 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599ff9bf95-fjqv6_7866138c-9592-4ed3-89a5-df4363514b42/kube-rbac-proxy/0.log" Jul 23 19:04:19.556517 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.556486 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599ff9bf95-fjqv6_7866138c-9592-4ed3-89a5-df4363514b42/prom-label-proxy/0.log" Jul 23 19:04:19.588838 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.588791 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599ff9bf95-fjqv6_7866138c-9592-4ed3-89a5-df4363514b42/kube-rbac-proxy-rules/0.log" Jul 23 19:04:19.617785 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:19.617697 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599ff9bf95-fjqv6_7866138c-9592-4ed3-89a5-df4363514b42/kube-rbac-proxy-metrics/0.log" Jul 23 19:04:21.022755 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.022696 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-rmj4r_5dfbf97b-129b-48f7-a0df-09148ecf8e6b/networking-console-plugin/0.log" Jul 23 19:04:21.903663 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.903625 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg"] Jul 23 19:04:21.910919 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.910887 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:21.929843 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.929065 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg"] Jul 23 19:04:21.956378 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.956344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-lib-modules\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:21.956552 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.956391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7d6z\" (UniqueName: \"kubernetes.io/projected/4cd8262a-90f1-4725-87c0-e4f5f77b7149-kube-api-access-p7d6z\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:21.956552 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.956507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-sys\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:21.956635 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.956561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-proc\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:21.956728 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:21.956708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-podres\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058222 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-podres\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058222 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-lib-modules\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7d6z\" (UniqueName: \"kubernetes.io/projected/4cd8262a-90f1-4725-87c0-e4f5f77b7149-kube-api-access-p7d6z\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-sys\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-proc\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058400 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-lib-modules\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058402 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-podres\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058448 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-proc\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.058804 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.058468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cd8262a-90f1-4725-87c0-e4f5f77b7149-sys\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.067089 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.067059 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7d6z\" (UniqueName: \"kubernetes.io/projected/4cd8262a-90f1-4725-87c0-e4f5f77b7149-kube-api-access-p7d6z\") pod \"perf-node-gather-daemonset-k6lpg\" (UID: \"4cd8262a-90f1-4725-87c0-e4f5f77b7149\") " pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.229735 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.229640 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:22.357653 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.357634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bf64c7869-vnw4l_40637513-b377-4170-8927-9982d61d3696/console/0.log" Jul 23 19:04:22.424333 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:22.423742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg"] Jul 23 19:04:23.150062 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:23.149114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" event={"ID":"4cd8262a-90f1-4725-87c0-e4f5f77b7149","Type":"ContainerStarted","Data":"df13dbad2de338d1b62a3a15e33971b992bb857eb636a7f3a6a61041b101416a"} Jul 23 19:04:23.150062 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:23.149184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" event={"ID":"4cd8262a-90f1-4725-87c0-e4f5f77b7149","Type":"ContainerStarted","Data":"ab21bcdd181e0c0b4e634c0f2a667f1cd17ff8ec2ade9f9b8019ed7f5cf36e16"} Jul 23 19:04:23.150062 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:23.150000 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:23.167567 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:23.167508 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" podStartSLOduration=2.167488677 podStartE2EDuration="2.167488677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:04:23.167014301 +0000 UTC m=+815.748158522" watchObservedRunningTime="2026-07-23 19:04:23.167488677 +0000 UTC m=+815.748632899" Jul 23 19:04:24.273482 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:24.273413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bz472_9b992573-9754-42a5-af40-5aedcc85454b/dns/0.log" Jul 23 19:04:24.304454 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:24.304412 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bz472_9b992573-9754-42a5-af40-5aedcc85454b/kube-rbac-proxy/0.log" Jul 23 19:04:24.395176 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:24.395149 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8dcd6_237ad0b3-52b5-4fc2-822d-4db074fb84aa/dns-node-resolver/0.log" Jul 23 19:04:25.121099 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:25.121075 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xlhsn_a57ae30e-7a34-4d0e-84f3-90be4fe65f97/node-ca/0.log" Jul 23 19:04:26.146949 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:26.146921 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48svgj_027ae2f8-e98c-4668-870a-90d35ce919ed/istio-proxy/0.log" Jul 23 19:04:26.239862 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:26.239812 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8f784977d-7fbz7_75dc15d6-0580-4360-97f0-a049f8851c2e/kube-auth-proxy/0.log" Jul 23 19:04:26.306488 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:26.306456 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/1.log" Jul 23 19:04:26.306663 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:26.306547 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/0.log" Jul 23 19:04:27.154017 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:27.153983 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d6htd_f387e34b-b873-48e9-8642-f31657c6309a/serve-healthcheck-canary/0.log" Jul 23 19:04:27.845945 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:27.845921 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tghlx_8f0b072b-a731-4848-8feb-14c367e5f882/kube-rbac-proxy/0.log" Jul 23 19:04:27.874036 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:27.874010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tghlx_8f0b072b-a731-4848-8feb-14c367e5f882/exporter/0.log" Jul 23 19:04:27.903665 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:27.903634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tghlx_8f0b072b-a731-4848-8feb-14c367e5f882/extractor/0.log" Jul 23 19:04:30.174562 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.174534 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ckwzf/perf-node-gather-daemonset-k6lpg" Jul 23 19:04:30.339547 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.339521 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dtwnv_3dcf84a9-0697-4547-a309-db90a958dca6/manager/0.log" Jul 23 19:04:30.379369 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.379324 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jrfwz_a644b371-42b3-4fb0-a9c8-4ff7c2f71762/manager/0.log" Jul 23 19:04:30.456096 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.456020 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-29h4l_2fb4eafe-cd1c-41b5-9b63-41ba326d0cb6/server/0.log" Jul 23 19:04:30.519431 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.519398 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nqrct_01252b47-6b6d-44ee-95c7-57d8e0fd1615/manager/0.log" Jul 23 19:04:30.713469 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.713385 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c495ff4d9-lpxjj_5e0089e0-fc30-4c90-9a19-2978a6883436/manager/0.log" Jul 23 19:04:30.745372 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:30.745344 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c495ff4d9-lpxjj_5e0089e0-fc30-4c90-9a19-2978a6883436/copy-manifests/0.log" Jul 23 19:04:32.139840 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:32.139786 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qh2gj_30d98f8f-f05f-4ca5-b860-0d0c9ab8a250/openshift-lws-operator/0.log" Jul 23 19:04:32.247326 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:32.247239 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48svgj_027ae2f8-e98c-4668-870a-90d35ce919ed/istio-proxy/0.log" Jul 23 19:04:32.316839 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:32.316793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8f784977d-7fbz7_75dc15d6-0580-4360-97f0-a049f8851c2e/kube-auth-proxy/0.log" Jul 23 19:04:32.356389 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:32.356284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/1.log" Jul 23 19:04:32.356619 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:32.356601 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/0.log" Jul 23 19:04:33.626937 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:33.626896 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48svgj_027ae2f8-e98c-4668-870a-90d35ce919ed/istio-proxy/0.log" Jul 23 19:04:33.733867 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:33.733836 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8f784977d-7fbz7_75dc15d6-0580-4360-97f0-a049f8851c2e/kube-auth-proxy/0.log" Jul 23 19:04:33.843474 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:33.843443 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/0.log" Jul 23 19:04:33.843474 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:33.843452 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-js5r6_c9d5bf26-fb59-4dfd-9181-1ad7611412e1/istio-proxy/1.log" Jul 23 19:04:39.115329 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.115297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8gshw_8cf5a5e7-5d1d-4847-a01d-41e3babe2687/kube-multus/0.log" Jul 23 19:04:39.177637 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.177609 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/kube-multus-additional-cni-plugins/0.log" Jul 23 19:04:39.208671 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.208642 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/egress-router-binary-copy/0.log" Jul 23 19:04:39.239977 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.239953 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/cni-plugins/0.log" Jul 23 19:04:39.265434 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.265407 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/bond-cni-plugin/0.log" Jul 23 19:04:39.305047 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.305018 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/routeoverride-cni/0.log" Jul 23 19:04:39.367365 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.367287 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/whereabouts-cni-bincopy/0.log" Jul 23 19:04:39.423708 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:39.423673 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7h7t6_11ac8e55-c6dc-4da3-a212-5cc7d3c74b96/whereabouts-cni/0.log" Jul 23 19:04:40.177922 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:40.177885 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zbwbh_bfb1861c-a4c3-4224-87ba-df1e74c3df08/network-metrics-daemon/0.log" Jul 23 19:04:40.203884 ip-10-0-141-105 kubenswrapper[2578]: I0723 19:04:40.203853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zbwbh_bfb1861c-a4c3-4224-87ba-df1e74c3df08/kube-rbac-proxy/0.log"