Jul 23 20:59:07.701535 ip-10-0-129-43 systemd[1]: Starting Kubernetes Kubelet... Jul 23 20:59:09.819154 ip-10-0-129-43 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 20:59:09.819154 ip-10-0-129-43 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 20:59:09.819154 ip-10-0-129-43 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 20:59:09.819154 ip-10-0-129-43 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 20:59:09.819154 ip-10-0-129-43 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 20:59:09.821469 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.821337 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 20:59:09.825402 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825386 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 20:59:09.825402 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825403 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825406 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825410 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825413 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825416 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825419 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825422 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825424 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825427 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825430 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825433 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825436 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825439 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825442 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825445 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825448 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825452 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825454 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825458 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 20:59:09.825471 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825461 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825464 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825467 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825470 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825473 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825476 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825479 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825481 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825484 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825487 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825490 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825492 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825495 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825497 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825500 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825503 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825505 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825508 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825511 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825513 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 20:59:09.825932 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825515 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825518 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825520 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825523 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825525 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825528 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825530 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825533 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825536 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825538 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825541 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825557 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825559 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825562 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825566 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825569 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825571 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825574 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825579 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 20:59:09.826443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825582 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825587 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825590 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825594 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825596 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825600 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825602 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825605 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825607 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825611 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825613 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825618 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825621 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825624 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825626 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825629 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825632 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825635 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825637 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825640 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 20:59:09.826913 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825642 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825644 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825648 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825653 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825655 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825658 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825660 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825663 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825666 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825669 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825671 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825674 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825677 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825680 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825682 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825685 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825687 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825689 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825692 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.825694 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 20:59:09.827369 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826410 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826417 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826420 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826423 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826426 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826429 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826431 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826434 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826437 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826440 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826443 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826446 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826448 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826451 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826454 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826456 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826459 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826461 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826464 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826468 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 20:59:09.827874 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826471 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826473 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826476 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826481 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826485 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826488 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826492 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826495 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826499 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826502 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826504 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826507 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826509 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826512 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826517 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826519 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826522 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826524 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826527 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 20:59:09.828387 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826530 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826534 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826537 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826540 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826556 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826559 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826562 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826565 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826568 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826572 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826575 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826578 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826582 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826585 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826588 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826590 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826593 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826597 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826600 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826603 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 20:59:09.828873 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826605 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826608 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826610 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826613 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826615 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826618 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826620 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826623 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826625 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826628 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826630 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826633 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826636 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826638 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826641 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826644 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826646 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826649 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826651 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 20:59:09.829336 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826654 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826656 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826659 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826662 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826664 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826668 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826671 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826673 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826676 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826678 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826681 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826683 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826686 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826688 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826691 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826694 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826697 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826699 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826702 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826704 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 20:59:09.829798 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.826707 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827096 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827101 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827104 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827107 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827110 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827112 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827115 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827118 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827121 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827123 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827126 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827129 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827131 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827134 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827136 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827139 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827142 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827145 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827148 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 20:59:09.830283 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827150 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827153 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827156 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827159 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827161 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827164 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827166 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827169 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827172 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827174 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827177 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827182 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827185 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827188 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827191 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827193 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827195 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827198 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827201 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 20:59:09.830763 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827203 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827206 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827209 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827211 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827214 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827217 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827219 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827222 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827224 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827227 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827231 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827233 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827237 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827241 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827243 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827246 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827249 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827251 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827254 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 20:59:09.831216 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827256 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827259 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827261 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827264 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827266 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827269 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827272 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827274 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827276 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827279 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827281 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827284 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827286 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827289 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827292 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827294 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827297 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827299 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827301 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827304 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 20:59:09.831684 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827306 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827309 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827314 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827317 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827320 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827322 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827325 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827329 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827331 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827335 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827338 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827341 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827344 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827347 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827349 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827352 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827354 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827357 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827359 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 20:59:09.832148 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827362 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827364 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.827367 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827443 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827451 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827457 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827462 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827467 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827470 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827474 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827479 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827482 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827485 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827488 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827494 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827498 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827501 2578 flags.go:64] FLAG: --cgroup-root="" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827504 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827507 2578 flags.go:64] FLAG: --client-ca-file="" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827510 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827514 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827518 2578 flags.go:64] FLAG: --cluster-domain="" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827520 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827524 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 20:59:09.832610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827527 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827530 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827534 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827537 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827540 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827559 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827564 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827568 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827573 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827576 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827580 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827584 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827587 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827590 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827593 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827596 2578 flags.go:64] FLAG: --enable-server="true" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827599 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827604 2578 flags.go:64] FLAG: --event-burst="100" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827607 2578 flags.go:64] FLAG: --event-qps="50" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827610 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827613 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827616 2578 flags.go:64] FLAG: --eviction-hard="" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827621 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827625 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827628 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 20:59:09.833182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827631 2578 flags.go:64] FLAG: --eviction-soft="" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827634 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827637 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827640 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827643 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827646 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827649 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827652 2578 flags.go:64] FLAG: --feature-gates="" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827656 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827659 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827662 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827666 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827669 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827672 2578 flags.go:64] FLAG: --help="false" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827675 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-129-43.ec2.internal" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827678 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827681 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827684 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827687 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827690 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827693 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827696 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827699 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 20:59:09.833850 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827702 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827705 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827708 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827711 2578 flags.go:64] FLAG: --kube-reserved="" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827714 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827716 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827721 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827725 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827728 2578 flags.go:64] FLAG: --lock-file="" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827731 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827734 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827737 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827742 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827744 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827748 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827750 2578 flags.go:64] FLAG: --logging-format="text" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827753 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827757 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827759 2578 flags.go:64] FLAG: --manifest-url="" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827762 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827767 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827770 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827774 2578 flags.go:64] FLAG: --max-pods="110" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827777 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827780 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 20:59:09.834459 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827783 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827785 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827788 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827792 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827795 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827803 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827806 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827809 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827812 2578 flags.go:64] FLAG: --pod-cidr="" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827815 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827821 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827824 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827828 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827832 2578 flags.go:64] FLAG: --port="10250" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827835 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827840 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-011b68f15d32cb773" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827843 2578 flags.go:64] FLAG: --qos-reserved="" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827846 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827849 2578 flags.go:64] FLAG: --register-node="true" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827851 2578 flags.go:64] FLAG: --register-with-taints="" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827855 2578 flags.go:64] FLAG: --registry-burst="10" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827858 2578 flags.go:64] FLAG: --registry-qps="5" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827861 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827864 2578 flags.go:64] FLAG: --reserved-memory="" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827868 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 20:59:09.835075 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827871 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827874 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827877 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827880 2578 flags.go:64] FLAG: --runonce="false" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827883 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827886 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827890 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827892 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827895 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827898 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827901 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827904 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827906 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827909 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827912 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827915 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827918 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827924 2578 flags.go:64] FLAG: --system-cgroups="" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827927 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827933 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827936 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827939 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827944 2578 flags.go:64] FLAG: --tls-min-version="" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827947 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827950 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 20:59:09.835686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827953 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827956 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827959 2578 flags.go:64] FLAG: --v="2" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827963 2578 flags.go:64] FLAG: --version="false" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827971 2578 flags.go:64] FLAG: --vmodule="" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827975 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.827978 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828098 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828102 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828105 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828107 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828110 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828114 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828117 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828120 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828166 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828171 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828175 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828178 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828181 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828183 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 20:59:09.836280 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828186 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828189 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828192 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828195 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828198 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828201 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828204 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828206 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828210 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828213 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828216 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828219 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828221 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828224 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828227 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828229 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828232 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828235 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828238 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 20:59:09.836844 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828240 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828243 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828247 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828251 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828254 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828257 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828259 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828262 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828265 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828267 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828270 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828272 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828275 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828277 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828280 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828283 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828285 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828288 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828292 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 20:59:09.837349 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828296 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828299 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828303 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828306 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828308 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828311 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828313 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828316 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828318 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828321 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828324 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828327 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828331 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828333 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828336 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828339 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828342 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828344 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828347 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828349 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 20:59:09.837831 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828352 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828354 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828357 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828359 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828362 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828364 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828367 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828369 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828372 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828374 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828376 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828379 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828382 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828385 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828389 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828392 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828395 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828397 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828400 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828403 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 20:59:09.838324 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828405 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828408 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828411 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828413 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828416 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828418 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.828421 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.829045 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.838093 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.838115 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.838137 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.838144 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838202 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838208 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838212 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 20:59:09.838879 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838215 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838218 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838221 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838224 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838227 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838230 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838233 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838235 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838238 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838240 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838243 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838246 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838249 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838260 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838263 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838266 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838268 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838271 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838274 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838276 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838279 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 20:59:09.839264 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838281 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838284 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838286 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838289 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838291 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838297 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838302 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838306 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838309 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838312 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838315 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838317 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838320 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838322 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838325 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838328 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838332 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838334 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838337 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 20:59:09.839776 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838340 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838343 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838346 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838349 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838351 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838354 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838357 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838359 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838362 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838365 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838367 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838370 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838372 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838375 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838377 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838380 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838382 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838385 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838389 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838393 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 20:59:09.840238 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838395 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838398 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838401 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838403 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838405 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838408 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838411 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838413 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838416 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838419 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838421 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838424 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838428 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838432 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838436 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838439 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838442 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838445 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838448 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 20:59:09.840944 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838450 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838453 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838455 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838459 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838462 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838464 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838467 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838470 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838472 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838475 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838478 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838481 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838483 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838487 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838490 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838493 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 20:59:09.841632 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838495 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.838500 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838650 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838658 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838661 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838664 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838667 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838670 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838674 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838677 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838679 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838681 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838684 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838687 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838689 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 20:59:09.842044 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838692 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838695 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838697 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838700 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838702 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838705 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838708 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838710 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838712 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838715 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838717 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838720 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838723 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838726 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838729 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838732 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838734 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838737 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838740 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838743 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 20:59:09.842420 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838745 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838748 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838750 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838753 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838756 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838758 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838761 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838763 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838766 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838768 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838771 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838773 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838776 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838778 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838781 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838784 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838786 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838789 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838791 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838794 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 20:59:09.842998 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838797 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838799 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838802 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838804 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838807 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838810 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838813 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838815 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838818 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838821 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838823 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838826 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838829 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838833 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838836 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838840 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838844 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838846 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838849 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 20:59:09.843511 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838852 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838854 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838857 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838859 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838862 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838865 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838867 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838870 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838873 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838876 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838878 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838882 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838884 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838887 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838890 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838893 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838895 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838898 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838901 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 20:59:09.844013 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838904 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838906 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838909 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838912 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838914 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838917 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838920 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:09.838923 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.838928 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.839090 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 20:59:09.844473 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.843326 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 20:59:09.844891 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.844662 2578 server.go:1019] "Starting client certificate rotation" Jul 23 20:59:09.844891 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.844760 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 20:59:09.844891 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.844796 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 20:59:09.847044 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.847026 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 20:59:09.850422 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.850406 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 20:59:09.860918 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.860887 2578 log.go:25] "Validated CRI v1 runtime API" Jul 23 20:59:09.863269 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.863246 2578 log.go:25] "Validated CRI v1 image API" Jul 23 20:59:09.864240 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.864224 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 20:59:09.868432 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.868401 2578 fs.go:135] Filesystem UUIDs: map[59b69d72-43f5-4a3c-ae40-68172da4872a:/dev/nvme0n1p4 5efc39f6-8519-45fc-aa6f-ee2b6af4c4f2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 23 20:59:09.868515 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.868429 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 20:59:09.872952 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.872827 2578 manager.go:216] Machine: {Timestamp:2026-07-23 20:59:09.872315338 +0000 UTC m=+0.862861870 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106094 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2214c31e164d3a9740b8d6d09d110d SystemUUID:ec2214c3-1e16-4d3a-9740-b8d6d09d110d BootID:31202ea6-eebb-458a-b910-c5653b2ced1a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:66:8a:29:15:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:66:8a:29:15:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:04:cb:3a:16:71 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 20:59:09.872952 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.872939 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 20:59:09.873092 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.873034 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 20:59:09.876077 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.876036 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 20:59:09.876224 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.876076 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-43.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 20:59:09.876836 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.876826 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 20:59:09.876869 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.876839 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 20:59:09.876869 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.876847 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 20:59:09.876869 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.876857 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 20:59:09.877051 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.877035 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 20:59:09.878090 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.878077 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 20:59:09.880980 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.880964 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 20:59:09.881158 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.881147 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 20:59:09.881195 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.881178 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 23 20:59:09.881226 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.881209 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 20:59:09.881254 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.881240 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 20:59:09.881492 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.881451 2578 kubelet.go:398] "Adding apiserver pod source" Jul 23 20:59:09.881538 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.881503 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 20:59:09.883264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.883243 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 20:59:09.883394 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.883282 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 20:59:09.886601 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.886575 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 20:59:09.887973 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.887958 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 20:59:09.888026 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.887990 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 20:59:09.888617 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888603 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 20:59:09.888617 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888620 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888626 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888631 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888639 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888645 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888680 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888685 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888692 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888698 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888706 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888714 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888737 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 20:59:09.888733 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.888742 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 20:59:09.892110 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.892092 2578 server.go:1268] "Started kubelet" Jul 23 20:59:09.892225 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.892192 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 20:59:09.892400 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.892348 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 20:59:09.892472 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.892426 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 20:59:09.893079 ip-10-0-129-43 systemd[1]: Started Kubernetes Kubelet. Jul 23 20:59:09.893610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.893404 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-43.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 20:59:09.893610 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.893560 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-43.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 20:59:09.893610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.893583 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 20:59:09.893781 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.893642 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 20:59:09.894925 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.894908 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 23 20:59:09.899105 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.899082 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 20:59:09.899899 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.899876 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 20:59:09.899899 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.899883 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 20:59:09.900045 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.899989 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:09.900524 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900500 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 20:59:09.900524 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900520 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 20:59:09.900705 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900503 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 20:59:09.900705 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900636 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 23 20:59:09.900705 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900646 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 23 20:59:09.900845 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900722 2578 factory.go:55] Registering systemd factory Jul 23 20:59:09.900845 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.900740 2578 factory.go:223] Registration of the systemd container factory successfully Jul 23 20:59:09.901145 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.901116 2578 factory.go:153] Registering CRI-O factory Jul 23 20:59:09.901145 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.901137 2578 factory.go:223] Registration of the crio container factory successfully Jul 23 20:59:09.901282 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.901193 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 20:59:09.901282 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.901220 2578 factory.go:103] Registering Raw factory Jul 23 20:59:09.901282 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.901233 2578 manager.go:1196] Started watching for new ooms in manager Jul 23 20:59:09.901727 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.901711 2578 manager.go:349] Starting recovery of all containers Jul 23 20:59:09.904410 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.904371 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-43.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 20:59:09.904500 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.904478 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 20:59:09.906494 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.904305 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-43.ec2.internal.18c506d77c19adad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-43.ec2.internal,UID:ip-10-0-129-43.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-43.ec2.internal,},FirstTimestamp:2026-07-23 20:59:09.892062637 +0000 UTC m=+0.882609152,LastTimestamp:2026-07-23 20:59:09.892062637 +0000 UTC m=+0.882609152,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-43.ec2.internal,}" Jul 23 20:59:09.913173 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.913118 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 20:59:09.915305 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.915282 2578 manager.go:354] Recovery completed Jul 23 20:59:09.920440 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.920419 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:09.922514 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.922494 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t8c7n" Jul 23 20:59:09.922897 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.922877 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:09.922897 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.922908 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:09.923012 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.922919 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:09.923466 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.923450 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 20:59:09.923466 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.923463 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 20:59:09.923597 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.923478 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:09.923597 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.923480 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 20:59:09.924319 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.924303 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:09.924374 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.924333 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:09.924374 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.924343 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:09.924896 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.924879 2578 policy_none.go:49] "None policy: Start" Jul 23 20:59:09.924933 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.924903 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 20:59:09.924933 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.924918 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 20:59:09.925130 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.925048 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-43.ec2.internal.18c506d77df023c4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-43.ec2.internal,UID:ip-10-0-129-43.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-43.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-43.ec2.internal,},FirstTimestamp:2026-07-23 20:59:09.922894788 +0000 UTC m=+0.913441302,LastTimestamp:2026-07-23 20:59:09.922894788 +0000 UTC m=+0.913441302,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-43.ec2.internal,}" Jul 23 20:59:09.926060 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.926041 2578 policy_none.go:47] "Start" Jul 23 20:59:09.930502 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.930327 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t8c7n" Jul 23 20:59:09.936845 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.936764 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-43.ec2.internal.18c506d77df06b56 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-43.ec2.internal,UID:ip-10-0-129-43.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-43.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-43.ec2.internal,},FirstTimestamp:2026-07-23 20:59:09.92291311 +0000 UTC m=+0.913459624,LastTimestamp:2026-07-23 20:59:09.92291311 +0000 UTC m=+0.913459624,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-43.ec2.internal,}" Jul 23 20:59:09.964343 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.964318 2578 manager.go:337] "Starting Device Plugin manager" Jul 23 20:59:09.964501 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.964371 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 20:59:09.964501 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.964384 2578 server.go:87] "Starting device plugin registration server" Jul 23 20:59:09.964750 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.964735 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 20:59:09.964823 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.964753 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 20:59:09.965415 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.965388 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 20:59:09.965610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.965496 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 20:59:09.965610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:09.965507 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 20:59:09.965721 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.965619 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 20:59:09.965721 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:09.965667 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.002532 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.002505 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 20:59:10.003834 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.002566 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 20:59:10.003834 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.002600 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 20:59:10.003834 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.002636 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 20:59:10.004945 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.004925 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 20:59:10.065705 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.065670 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:10.066721 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.066704 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:10.066821 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.066736 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:10.066821 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.066749 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:10.066821 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.066774 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.075864 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.075834 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.075978 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.075870 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-43.ec2.internal\": node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.101795 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.101757 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.102750 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.102725 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal"] Jul 23 20:59:10.102811 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.102795 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:10.104376 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.104358 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:10.104502 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.104388 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:10.104502 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.104397 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:10.105691 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.105675 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:10.105834 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.105818 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.105892 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.105854 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:10.106564 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.106530 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:10.106667 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.106575 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:10.106667 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.106588 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:10.106667 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.106589 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:10.106667 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.106614 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:10.106667 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.106627 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:10.107918 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.107902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.107995 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.107944 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:10.108694 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.108677 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:10.108796 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.108704 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:10.108796 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.108716 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:10.136336 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.136310 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-43.ec2.internal\" not found" node="ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.141921 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.141896 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-43.ec2.internal\" not found" node="ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.200930 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.200902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28b10fa3f906b551eed3bc9b0e2f57f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal\" (UID: \"28b10fa3f906b551eed3bc9b0e2f57f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.200930 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.200935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28b10fa3f906b551eed3bc9b0e2f57f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal\" (UID: \"28b10fa3f906b551eed3bc9b0e2f57f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.201097 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.200954 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07744b71299bb4e3a3b21c8d453b608e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-43.ec2.internal\" (UID: \"07744b71299bb4e3a3b21c8d453b608e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.201847 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.201818 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.301212 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.301139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28b10fa3f906b551eed3bc9b0e2f57f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal\" (UID: \"28b10fa3f906b551eed3bc9b0e2f57f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.301212 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.301174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28b10fa3f906b551eed3bc9b0e2f57f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal\" (UID: \"28b10fa3f906b551eed3bc9b0e2f57f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.301212 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.301191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07744b71299bb4e3a3b21c8d453b608e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-43.ec2.internal\" (UID: \"07744b71299bb4e3a3b21c8d453b608e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.301348 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.301237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07744b71299bb4e3a3b21c8d453b608e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-43.ec2.internal\" (UID: \"07744b71299bb4e3a3b21c8d453b608e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.301348 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.301250 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28b10fa3f906b551eed3bc9b0e2f57f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal\" (UID: \"28b10fa3f906b551eed3bc9b0e2f57f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.301348 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.301264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28b10fa3f906b551eed3bc9b0e2f57f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal\" (UID: \"28b10fa3f906b551eed3bc9b0e2f57f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.302203 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.302188 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.402955 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.402926 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.439128 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.439102 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.446390 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.446369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" Jul 23 20:59:10.503509 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.503470 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.603979 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.603898 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.704430 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.704394 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.804921 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.804882 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.844070 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.844046 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 20:59:10.844506 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.844189 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 20:59:10.905010 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:10.904935 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:10.935429 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.935379 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 20:54:09 +0000 UTC" deadline="2028-02-14 15:02:24.917365031 +0000 UTC" Jul 23 20:59:10.935429 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.935416 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13698h3m13.981952372s" Jul 23 20:59:10.956502 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:10.956467 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b10fa3f906b551eed3bc9b0e2f57f1.slice/crio-317dd704dcb46ae297fb1c5e896ea47815cfcae9cf116d7b37a979a1e602c2d6 WatchSource:0}: Error finding container 317dd704dcb46ae297fb1c5e896ea47815cfcae9cf116d7b37a979a1e602c2d6: Status 404 returned error can't find the container with id 317dd704dcb46ae297fb1c5e896ea47815cfcae9cf116d7b37a979a1e602c2d6 Jul 23 20:59:10.956893 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:10.956868 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07744b71299bb4e3a3b21c8d453b608e.slice/crio-3de636bdb97b454dfcda8c2f7d188e586a628303fa26445dd03c28ca06a9d8f3 WatchSource:0}: Error finding container 3de636bdb97b454dfcda8c2f7d188e586a628303fa26445dd03c28ca06a9d8f3: Status 404 returned error can't find the container with id 3de636bdb97b454dfcda8c2f7d188e586a628303fa26445dd03c28ca06a9d8f3 Jul 23 20:59:10.960697 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:10.960677 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 20:59:11.005002 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:11.004974 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:11.006179 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.006127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" event={"ID":"07744b71299bb4e3a3b21c8d453b608e","Type":"ContainerStarted","Data":"3de636bdb97b454dfcda8c2f7d188e586a628303fa26445dd03c28ca06a9d8f3"} Jul 23 20:59:11.007182 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.007157 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" event={"ID":"28b10fa3f906b551eed3bc9b0e2f57f1","Type":"ContainerStarted","Data":"317dd704dcb46ae297fb1c5e896ea47815cfcae9cf116d7b37a979a1e602c2d6"} Jul 23 20:59:11.073361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.073334 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 20:59:11.074694 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.074676 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientMemory" Jul 23 20:59:11.074788 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.074708 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 20:59:11.074788 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.074719 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeHasSufficientPID" Jul 23 20:59:11.105393 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:11.105355 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:11.205840 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:11.205751 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-43.ec2.internal\" not found" Jul 23 20:59:11.206014 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.205997 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 20:59:11.218985 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.218961 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 20:59:11.300897 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.300862 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" Jul 23 20:59:11.314076 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.314038 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 20:59:11.315813 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.315795 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" Jul 23 20:59:11.323919 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.323897 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 20:59:11.343965 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.343939 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 20:59:11.790813 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.790782 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 20:59:11.882617 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.882578 2578 apiserver.go:52] "Watching apiserver" Jul 23 20:59:11.889617 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.889591 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 20:59:11.890049 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.890018 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hlqpk","openshift-dns/node-resolver-72tl8","openshift-multus/multus-dqtbt","openshift-multus/network-metrics-daemon-6r9p8","openshift-network-operator/iptables-alerter-k82w9","openshift-ovn-kubernetes/ovnkube-node-mr549","kube-system/global-pull-secret-syncer-bs7mm","kube-system/konnectivity-agent-stjx7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt","openshift-image-registry/node-ca-d8j56","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal","openshift-multus/multus-additional-cni-plugins-n2cfh","openshift-network-diagnostics/network-check-target-vcsvx"] Jul 23 20:59:11.892111 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.892088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.894353 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.894310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.895399 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.895372 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 20:59:11.895706 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.895688 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.895819 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.895796 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 20:59:11.896181 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.896130 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 20:59:11.896258 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.896175 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hh77g\"" Jul 23 20:59:11.896450 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.896432 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.897094 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.897068 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 20:59:11.897253 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.897235 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 20:59:11.897399 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.897379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:11.899383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.898515 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-99t8k\"" Jul 23 20:59:11.899383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.898771 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 20:59:11.899383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.898876 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.899383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.898942 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 20:59:11.899383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.899021 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.899383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.899242 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.899843 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.899774 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.899896 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.899774 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.900045 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.900029 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 20:59:11.900200 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.900147 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wqvdg\"" Jul 23 20:59:11.900495 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.900435 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:11.900599 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:11.900527 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:11.901795 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.901727 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 20:59:11.902642 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.902450 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qzq7v\"" Jul 23 20:59:11.903305 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.903286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:11.904849 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.904829 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:11.905503 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.905473 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d7l5h\"" Jul 23 20:59:11.905616 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.905594 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 20:59:11.905698 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.905682 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.906103 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.906069 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.906840 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.906823 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:11.906999 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:11.906930 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:11.907101 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.907076 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qv97r\"" Jul 23 20:59:11.907152 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.907130 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 20:59:11.907202 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.907176 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 20:59:11.908139 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.908119 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:11.909899 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.909881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-system-cni-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.910033 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-kubelet\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.910154 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-log-socket\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.910276 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-cni-netd\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.910276 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.910517 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910493 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dmdt9\"" Jul 23 20:59:11.910606 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910517 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.910886 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.910763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:11.911042 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-env-overrides\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.911119 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911051 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-cni-multus\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.911119 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2306fe4d-4766-4135-834e-8f2d7e916321-multus-daemon-config\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.911119 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-etc-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.911264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-ovn\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.911264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911160 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-run-ovn-kubernetes\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.911264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911182 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-cni-bin\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.911264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.911264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-socket-dir-parent\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-netns\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-conf-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-systemd-units\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-system-cni-dir\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911425 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-cnibin\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911450 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6clz\" (UniqueName: \"kubernetes.io/projected/f3229463-d919-4499-9c87-61ab3de151ff-kube-api-access-m6clz\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df6d5abf-4234-467e-97c5-996ca00b98e3-iptables-alerter-script\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911578 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.911596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911582 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-os-release\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911605 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2306fe4d-4766-4135-834e-8f2d7e916321-cni-binary-copy\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-kubelet\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qncbp\" (UniqueName: \"kubernetes.io/projected/2306fe4d-4766-4135-834e-8f2d7e916321-kube-api-access-qncbp\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-cnibin\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-cni-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911766 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-multus-certs\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-systemd\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.911997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-node-log\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912022 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovn-node-metrics-cert\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovnkube-script-lib\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6kb\" (UniqueName: \"kubernetes.io/projected/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-kube-api-access-hm6kb\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912136 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-k8s-cni-cncf-io\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912181 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-cni-bin\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df6d5abf-4234-467e-97c5-996ca00b98e3-host-slash\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-slash\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-run-netns\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9v7v\" (UniqueName: \"kubernetes.io/projected/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-kube-api-access-w9v7v\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-hostroot\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-etc-kubernetes\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwb2s\" (UniqueName: \"kubernetes.io/projected/df6d5abf-4234-467e-97c5-996ca00b98e3-kube-api-access-zwb2s\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-var-lib-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912455 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovnkube-config\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912479 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-os-release\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912603 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8w4f\" (UniqueName: \"kubernetes.io/projected/549e7963-6203-406f-9c29-2716303002d5-kube-api-access-f8w4f\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:11.912736 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-hosts-file\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:11.913513 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.912686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-tmp-dir\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:11.913513 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.913424 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:11.913632 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:11.913498 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:11.914417 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.914393 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.914912 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.914866 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.915138 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.915105 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 20:59:11.915243 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.915171 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mhnll\"" Jul 23 20:59:11.915243 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.915234 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7flth\"" Jul 23 20:59:11.915359 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.915280 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 20:59:11.915581 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.915458 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 20:59:11.915581 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.915528 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 20:59:11.964185 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.964140 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-54g87" Jul 23 20:59:11.969636 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:11.969605 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-54g87" Jul 23 20:59:12.002210 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.002179 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 20:59:12.013669 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-os-release\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.013669 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013674 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-kubelet\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qncbp\" (UniqueName: \"kubernetes.io/projected/2306fe4d-4766-4135-834e-8f2d7e916321-kube-api-access-qncbp\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-registration-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-kubelet-config\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkxgp\" (UniqueName: \"kubernetes.io/projected/fe72a6e4-1476-406c-85c0-798df5aab117-kube-api-access-zkxgp\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013775 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-kubelet\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013788 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-os-release\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0213c346-df99-4945-9f79-884622623238-konnectivity-ca\") pod \"konnectivity-agent-stjx7\" (UID: \"0213c346-df99-4945-9f79-884622623238\") " pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92ec2b16-a339-487b-9f50-8f58b9e1825b-tmp\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.013903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-systemd\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013933 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-systemd\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.013973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-node-log\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovn-node-metrics-cert\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovnkube-script-lib\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-k8s-cni-cncf-io\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014070 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-node-log\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-host\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-run-netns\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9v7v\" (UniqueName: \"kubernetes.io/projected/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-kube-api-access-w9v7v\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-hostroot\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.014245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-etc-kubernetes\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-modprobe-d\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014299 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-var-lib-kubelet\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-var-lib-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovnkube-config\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8w4f\" (UniqueName: \"kubernetes.io/projected/549e7963-6203-406f-9c29-2716303002d5-kube-api-access-f8w4f\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-var-lib-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-tmp-dir\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014485 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-etc-kubernetes\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysconfig\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014435 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-k8s-cni-cncf-io\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014587 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-run-netns\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.014722 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014623 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-hostroot\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014742 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-tmp-dir\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovnkube-script-lib\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysctl-conf\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014824 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-kubelet\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-cni-netd\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.014988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovnkube-config\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015000 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-kubelet\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-env-overrides\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015072 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-cni-netd\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2306fe4d-4766-4135-834e-8f2d7e916321-multus-daemon-config\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015112 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-cni-bin\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015261 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-conf-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6clz\" (UniqueName: \"kubernetes.io/projected/f3229463-d919-4499-9c87-61ab3de151ff-kube-api-access-m6clz\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-sys-fs\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9kfz\" (UniqueName: \"kubernetes.io/projected/b22b0230-000b-4fd7-8596-b5832724cecb-kube-api-access-c9kfz\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-cnibin\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015572 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-env-overrides\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2306fe4d-4766-4135-834e-8f2d7e916321-cni-binary-copy\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015619 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-cnibin\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015664 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-cnibin\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015665 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-cni-bin\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2306fe4d-4766-4135-834e-8f2d7e916321-multus-daemon-config\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015715 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-conf-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.015724 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015755 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-cni-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.015971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015757 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-cnibin\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.015793 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 20:59:12.515774606 +0000 UTC m=+3.506321132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-multus-certs\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015858 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-kubernetes\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6kb\" (UniqueName: \"kubernetes.io/projected/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-kube-api-access-hm6kb\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-cni-bin\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015931 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df6d5abf-4234-467e-97c5-996ca00b98e3-host-slash\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015947 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-cni-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.015952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwb2s\" (UniqueName: \"kubernetes.io/projected/df6d5abf-4234-467e-97c5-996ca00b98e3-kube-api-access-zwb2s\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016032 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016090 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df6d5abf-4234-467e-97c5-996ca00b98e3-host-slash\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-socket-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016125 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-multus-certs\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016170 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-cni-bin\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.016861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-device-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysctl-d\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-slash\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016278 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-slash\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0213c346-df99-4945-9f79-884622623238-agent-certs\") pod \"konnectivity-agent-stjx7\" (UID: \"0213c346-df99-4945-9f79-884622623238\") " pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-os-release\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2306fe4d-4766-4135-834e-8f2d7e916321-cni-binary-copy\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-os-release\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-hosts-file\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-system-cni-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-tuned\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkhpc\" (UniqueName: \"kubernetes.io/projected/92ec2b16-a339-487b-9f50-8f58b9e1825b-kube-api-access-mkhpc\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016528 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-log-socket\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-cni-multus\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016594 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-system-cni-dir\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016610 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-var-lib-cni-multus\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.017621 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-log-socket\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016632 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-hosts-file\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-systemd\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b22b0230-000b-4fd7-8596-b5832724cecb-host\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-etc-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-ovn\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-etc-openvswitch\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-run-ovn-kubernetes\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016856 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-run-ovn\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016910 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-host-run-ovn-kubernetes\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-socket-dir-parent\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-netns\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.016994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-host-run-netns\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2306fe4d-4766-4135-834e-8f2d7e916321-multus-socket-dir-parent\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.018443 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df6d5abf-4234-467e-97c5-996ca00b98e3-iptables-alerter-script\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-systemd-units\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-system-cni-dir\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-dbus\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017167 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-systemd-units\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017173 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/549e7963-6203-406f-9c29-2716303002d5-system-cni-dir\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-run\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-sys\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-lib-modules\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b22b0230-000b-4fd7-8596-b5832724cecb-serviceca\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/549e7963-6203-406f-9c29-2716303002d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.017481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df6d5abf-4234-467e-97c5-996ca00b98e3-iptables-alerter-script\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.019306 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.019017 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-ovn-node-metrics-cert\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.021912 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.021882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qncbp\" (UniqueName: \"kubernetes.io/projected/2306fe4d-4766-4135-834e-8f2d7e916321-kube-api-access-qncbp\") pod \"multus-dqtbt\" (UID: \"2306fe4d-4766-4135-834e-8f2d7e916321\") " pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.025834 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.025779 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8w4f\" (UniqueName: \"kubernetes.io/projected/549e7963-6203-406f-9c29-2716303002d5-kube-api-access-f8w4f\") pod \"multus-additional-cni-plugins-n2cfh\" (UID: \"549e7963-6203-406f-9c29-2716303002d5\") " pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.026442 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.026420 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9v7v\" (UniqueName: \"kubernetes.io/projected/2b959712-8ea5-4d5c-97c8-c53de4c55ad9-kube-api-access-w9v7v\") pod \"node-resolver-72tl8\" (UID: \"2b959712-8ea5-4d5c-97c8-c53de4c55ad9\") " pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.027316 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.027295 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwb2s\" (UniqueName: \"kubernetes.io/projected/df6d5abf-4234-467e-97c5-996ca00b98e3-kube-api-access-zwb2s\") pod \"iptables-alerter-k82w9\" (UID: \"df6d5abf-4234-467e-97c5-996ca00b98e3\") " pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.027824 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.027803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6clz\" (UniqueName: \"kubernetes.io/projected/f3229463-d919-4499-9c87-61ab3de151ff-kube-api-access-m6clz\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:12.027904 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.027803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6kb\" (UniqueName: \"kubernetes.io/projected/a600fc50-247e-4a0d-8e7d-a7f3e5396cd1-kube-api-access-hm6kb\") pod \"ovnkube-node-mr549\" (UID: \"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.118685 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-modprobe-d\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.118685 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-var-lib-kubelet\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.118685 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysconfig\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysctl-conf\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-sys-fs\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9kfz\" (UniqueName: \"kubernetes.io/projected/b22b0230-000b-4fd7-8596-b5832724cecb-kube-api-access-c9kfz\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-modprobe-d\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-var-lib-kubelet\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.118919 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:12.118937 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118931 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysconfig\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.118977 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:12.618956745 +0000 UTC m=+3.609503249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119059 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysctl-conf\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119109 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-sys-fs\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.118813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-kubernetes\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-socket-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.119267 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-device-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysctl-d\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0213c346-df99-4945-9f79-884622623238-agent-certs\") pod \"konnectivity-agent-stjx7\" (UID: \"0213c346-df99-4945-9f79-884622623238\") " pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-tuned\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkhpc\" (UniqueName: \"kubernetes.io/projected/92ec2b16-a339-487b-9f50-8f58b9e1825b-kube-api-access-mkhpc\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-systemd\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b22b0230-000b-4fd7-8596-b5832724cecb-host\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119837 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-kubernetes\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119847 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-sysctl-d\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.119973 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120076 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-systemd\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-device-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120221 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b22b0230-000b-4fd7-8596-b5832724cecb-host\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-socket-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-dbus\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-run\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120589 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-sys\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-lib-modules\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-dbus\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b22b0230-000b-4fd7-8596-b5832724cecb-serviceca\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120762 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-lib-modules\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-registration-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-run\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120817 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-kubelet-config\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120824 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-sys\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.120977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120732 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.121378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkxgp\" (UniqueName: \"kubernetes.io/projected/fe72a6e4-1476-406c-85c0-798df5aab117-kube-api-access-zkxgp\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.121378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121120 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0213c346-df99-4945-9f79-884622623238-konnectivity-ca\") pod \"konnectivity-agent-stjx7\" (UID: \"0213c346-df99-4945-9f79-884622623238\") " pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.121378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92ec2b16-a339-487b-9f50-8f58b9e1825b-tmp\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.121378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-host\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.121378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121294 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b22b0230-000b-4fd7-8596-b5832724cecb-serviceca\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.121378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92ec2b16-a339-487b-9f50-8f58b9e1825b-host\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.124361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.122092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0213c346-df99-4945-9f79-884622623238-konnectivity-ca\") pod \"konnectivity-agent-stjx7\" (UID: \"0213c346-df99-4945-9f79-884622623238\") " pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.124361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.120974 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-kubelet-config\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.124361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.121055 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe72a6e4-1476-406c-85c0-798df5aab117-registration-dir\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.124361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.122990 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0213c346-df99-4945-9f79-884622623238-agent-certs\") pod \"konnectivity-agent-stjx7\" (UID: \"0213c346-df99-4945-9f79-884622623238\") " pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.126437 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.126415 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92ec2b16-a339-487b-9f50-8f58b9e1825b-etc-tuned\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.126524 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.126451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92ec2b16-a339-487b-9f50-8f58b9e1825b-tmp\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.128830 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.128806 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:12.128830 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.128833 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:12.128982 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.128848 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:12.128982 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.128922 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 20:59:12.628901898 +0000 UTC m=+3.619448417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:12.129104 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.129048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9kfz\" (UniqueName: \"kubernetes.io/projected/b22b0230-000b-4fd7-8596-b5832724cecb-kube-api-access-c9kfz\") pod \"node-ca-d8j56\" (UID: \"b22b0230-000b-4fd7-8596-b5832724cecb\") " pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.130565 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.130527 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkhpc\" (UniqueName: \"kubernetes.io/projected/92ec2b16-a339-487b-9f50-8f58b9e1825b-kube-api-access-mkhpc\") pod \"tuned-hlqpk\" (UID: \"92ec2b16-a339-487b-9f50-8f58b9e1825b\") " pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.130789 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.130770 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkxgp\" (UniqueName: \"kubernetes.io/projected/fe72a6e4-1476-406c-85c0-798df5aab117-kube-api-access-zkxgp\") pod \"aws-ebs-csi-driver-node-fdnrt\" (UID: \"fe72a6e4-1476-406c-85c0-798df5aab117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.205904 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.205865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:12.216996 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.216957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" Jul 23 20:59:12.227997 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.227968 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-72tl8" Jul 23 20:59:12.233724 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.233705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dqtbt" Jul 23 20:59:12.243264 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.243234 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k82w9" Jul 23 20:59:12.251124 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.251102 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:12.259207 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.259181 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" Jul 23 20:59:12.268110 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.268083 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" Jul 23 20:59:12.273756 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.273690 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d8j56" Jul 23 20:59:12.524234 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.524144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:12.524411 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.524266 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:12.524411 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.524329 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 20:59:13.524308746 +0000 UTC m=+4.514855249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:12.624697 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.624654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:12.624868 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.624832 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:12.624944 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.624910 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:13.624891808 +0000 UTC m=+4.615438310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:12.725315 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.725266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:12.725513 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.725455 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:12.725513 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.725482 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:12.725513 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.725494 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:12.725699 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:12.725566 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 20:59:13.725538268 +0000 UTC m=+4.716084770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:12.894443 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.894415 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2306fe4d_4766_4135_834e_8f2d7e916321.slice/crio-6f58706942ae77db74aa5c0df044409d6d779b588225e62209dd74932668c887 WatchSource:0}: Error finding container 6f58706942ae77db74aa5c0df044409d6d779b588225e62209dd74932668c887: Status 404 returned error can't find the container with id 6f58706942ae77db74aa5c0df044409d6d779b588225e62209dd74932668c887 Jul 23 20:59:12.895749 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.895721 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe72a6e4_1476_406c_85c0_798df5aab117.slice/crio-2e0e3b60d77cfb67b38bb10e2d5b7ee49d95f2aa2e091a5d37d70a1e9e3c8bb9 WatchSource:0}: Error finding container 2e0e3b60d77cfb67b38bb10e2d5b7ee49d95f2aa2e091a5d37d70a1e9e3c8bb9: Status 404 returned error can't find the container with id 2e0e3b60d77cfb67b38bb10e2d5b7ee49d95f2aa2e091a5d37d70a1e9e3c8bb9 Jul 23 20:59:12.897050 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.897008 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda600fc50_247e_4a0d_8e7d_a7f3e5396cd1.slice/crio-ba296e458bdc042e80131272493d4c78ca1a367407ac60ea20e72aa52e9dd590 WatchSource:0}: Error finding container ba296e458bdc042e80131272493d4c78ca1a367407ac60ea20e72aa52e9dd590: Status 404 returned error can't find the container with id ba296e458bdc042e80131272493d4c78ca1a367407ac60ea20e72aa52e9dd590 Jul 23 20:59:12.900322 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.900296 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ec2b16_a339_487b_9f50_8f58b9e1825b.slice/crio-00da4513392803d8e4ebb3948ba9ed1e3e9662e09eb9c2e256b92e49fd2bb372 WatchSource:0}: Error finding container 00da4513392803d8e4ebb3948ba9ed1e3e9662e09eb9c2e256b92e49fd2bb372: Status 404 returned error can't find the container with id 00da4513392803d8e4ebb3948ba9ed1e3e9662e09eb9c2e256b92e49fd2bb372 Jul 23 20:59:12.901496 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.901467 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6d5abf_4234_467e_97c5_996ca00b98e3.slice/crio-453f87a6b09ebc7bb63296a43a912786a96740ab9038b743547e4d6596014e05 WatchSource:0}: Error finding container 453f87a6b09ebc7bb63296a43a912786a96740ab9038b743547e4d6596014e05: Status 404 returned error can't find the container with id 453f87a6b09ebc7bb63296a43a912786a96740ab9038b743547e4d6596014e05 Jul 23 20:59:12.902362 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.902199 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b959712_8ea5_4d5c_97c8_c53de4c55ad9.slice/crio-b4d6ad6cf9d7849d1eb69b2f7f91a343ac7fc535add7435ae7d3e98c62e653ec WatchSource:0}: Error finding container b4d6ad6cf9d7849d1eb69b2f7f91a343ac7fc535add7435ae7d3e98c62e653ec: Status 404 returned error can't find the container with id b4d6ad6cf9d7849d1eb69b2f7f91a343ac7fc535add7435ae7d3e98c62e653ec Jul 23 20:59:12.905463 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:12.903355 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod549e7963_6203_406f_9c29_2716303002d5.slice/crio-51df71a21801446ff847a0b67b14c40966e3c1c607e12aca3b766e3a734f7982 WatchSource:0}: Error finding container 51df71a21801446ff847a0b67b14c40966e3c1c607e12aca3b766e3a734f7982: Status 404 returned error can't find the container with id 51df71a21801446ff847a0b67b14c40966e3c1c607e12aca3b766e3a734f7982 Jul 23 20:59:12.972315 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.972279 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 20:54:11 +0000 UTC" deadline="2028-04-21 01:56:00.248438982 +0000 UTC" Jul 23 20:59:12.972315 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:12.972311 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15292h56m47.276131825s" Jul 23 20:59:13.003867 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.003664 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:13.004004 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.003953 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:13.013613 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.013579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" event={"ID":"fe72a6e4-1476-406c-85c0-798df5aab117","Type":"ContainerStarted","Data":"2e0e3b60d77cfb67b38bb10e2d5b7ee49d95f2aa2e091a5d37d70a1e9e3c8bb9"} Jul 23 20:59:13.017691 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.017654 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dqtbt" event={"ID":"2306fe4d-4766-4135-834e-8f2d7e916321","Type":"ContainerStarted","Data":"6f58706942ae77db74aa5c0df044409d6d779b588225e62209dd74932668c887"} Jul 23 20:59:13.019610 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.019581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" event={"ID":"07744b71299bb4e3a3b21c8d453b608e","Type":"ContainerStarted","Data":"4fcae699184893168a221a35f4c8c32eed214199f81d64d356ada7ff00c026c2"} Jul 23 20:59:13.021107 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.021081 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-stjx7" event={"ID":"0213c346-df99-4945-9f79-884622623238","Type":"ContainerStarted","Data":"8a7daba879abb2d067e766be8d27b7cd7c3102e05f3d8f42d758a0482d4607a4"} Jul 23 20:59:13.023361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.023332 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d8j56" event={"ID":"b22b0230-000b-4fd7-8596-b5832724cecb","Type":"ContainerStarted","Data":"e4e386405419357efeb94a672991e127772045720e9cc1cebfd8ed32768f09c4"} Jul 23 20:59:13.024376 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.024346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-72tl8" event={"ID":"2b959712-8ea5-4d5c-97c8-c53de4c55ad9","Type":"ContainerStarted","Data":"b4d6ad6cf9d7849d1eb69b2f7f91a343ac7fc535add7435ae7d3e98c62e653ec"} Jul 23 20:59:13.026618 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.026591 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerStarted","Data":"51df71a21801446ff847a0b67b14c40966e3c1c607e12aca3b766e3a734f7982"} Jul 23 20:59:13.030110 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.030083 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k82w9" event={"ID":"df6d5abf-4234-467e-97c5-996ca00b98e3","Type":"ContainerStarted","Data":"453f87a6b09ebc7bb63296a43a912786a96740ab9038b743547e4d6596014e05"} Jul 23 20:59:13.031699 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.031671 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" event={"ID":"92ec2b16-a339-487b-9f50-8f58b9e1825b","Type":"ContainerStarted","Data":"00da4513392803d8e4ebb3948ba9ed1e3e9662e09eb9c2e256b92e49fd2bb372"} Jul 23 20:59:13.033166 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.033147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"ba296e458bdc042e80131272493d4c78ca1a367407ac60ea20e72aa52e9dd590"} Jul 23 20:59:13.042934 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.042892 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-43.ec2.internal" podStartSLOduration=2.042877521 podStartE2EDuration="2.042877521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:13.042565303 +0000 UTC m=+4.033111842" watchObservedRunningTime="2026-07-23 20:59:13.042877521 +0000 UTC m=+4.033424044" Jul 23 20:59:13.532183 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.531419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:13.532183 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.531691 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:13.532183 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.531762 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 20:59:15.531741286 +0000 UTC m=+6.522287790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:13.633126 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.632494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:13.633126 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.632717 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:13.633126 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.632780 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:15.632764364 +0000 UTC m=+6.623310869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:13.733776 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.733718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:13.733958 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.733915 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:13.733958 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.733938 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:13.733958 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.733951 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:13.734108 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:13.734009 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 20:59:15.733991325 +0000 UTC m=+6.724537827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:13.973282 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.973157 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 20:54:11 +0000 UTC" deadline="2028-02-08 19:16:14.382836786 +0000 UTC" Jul 23 20:59:13.973282 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:13.973219 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13558h17m0.409622488s" Jul 23 20:59:14.008032 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:14.004062 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:14.008032 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:14.004204 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:14.008032 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:14.005815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:14.008032 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:14.005916 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:14.046129 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:14.044859 2578 generic.go:356] "Generic (PLEG): container finished" podID="28b10fa3f906b551eed3bc9b0e2f57f1" containerID="fd44073f5209a44ef715aa5a65d3f76760f4d10751f699bd30dd0a9789896834" exitCode=0 Jul 23 20:59:14.046129 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:14.045906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" event={"ID":"28b10fa3f906b551eed3bc9b0e2f57f1","Type":"ContainerDied","Data":"fd44073f5209a44ef715aa5a65d3f76760f4d10751f699bd30dd0a9789896834"} Jul 23 20:59:15.003145 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:15.003109 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:15.003659 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.003240 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:15.067672 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:15.067628 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" event={"ID":"28b10fa3f906b551eed3bc9b0e2f57f1","Type":"ContainerStarted","Data":"c466858a4150acac5887d441050b18d9e58b05fc309ffd7f1bd3e532d50d816b"} Jul 23 20:59:15.084047 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:15.083988 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-43.ec2.internal" podStartSLOduration=4.08396482 podStartE2EDuration="4.08396482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:15.083729764 +0000 UTC m=+6.074276288" watchObservedRunningTime="2026-07-23 20:59:15.08396482 +0000 UTC m=+6.074511346" Jul 23 20:59:15.550401 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:15.550357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:15.550621 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.550603 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:15.550698 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.550686 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 20:59:19.55065731 +0000 UTC m=+10.541203816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:15.651367 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:15.651328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:15.651588 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.651487 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:15.651588 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.651572 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:19.651536369 +0000 UTC m=+10.642082885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:15.752821 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:15.752775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:15.752999 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.752935 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:15.752999 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.752955 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:15.752999 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.752968 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:15.753165 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:15.753026 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 20:59:19.753006004 +0000 UTC m=+10.743552523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:16.005626 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:16.005523 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:16.006077 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:16.005698 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:16.006155 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:16.006137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:16.006336 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:16.006303 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:17.004336 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:17.003690 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:17.004336 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:17.003821 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:18.005661 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:18.003977 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:18.005661 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:18.004134 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:18.005661 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:18.004632 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:18.005661 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:18.004750 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:19.002879 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:19.002831 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:19.003079 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.002965 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:19.586875 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:19.586835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:19.587344 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.587036 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:19.587344 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.587105 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 20:59:27.587086132 +0000 UTC m=+18.577632635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:19.688303 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:19.687599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:19.688303 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.687881 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:19.688303 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.687956 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:27.687937151 +0000 UTC m=+18.678483656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:19.788798 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:19.788166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:19.788798 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.788350 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:19.788798 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.788372 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:19.788798 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.788385 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:19.788798 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:19.788443 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 20:59:27.788423992 +0000 UTC m=+18.778970497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:20.006105 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:20.005617 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:20.006105 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:20.005664 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:20.006105 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:20.005738 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:20.006105 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:20.005871 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:21.003476 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:21.003438 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:21.003933 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:21.003582 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:22.003180 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:22.003141 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:22.003371 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:22.003298 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:22.003371 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:22.003349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:22.003480 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:22.003457 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:23.003655 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:23.003620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:23.004097 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:23.003732 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:24.003606 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:24.003564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:24.003606 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:24.003606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:24.004138 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:24.003706 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:24.004138 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:24.003850 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:25.003744 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:25.003703 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:25.003942 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:25.003857 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:26.003747 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:26.003708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:26.003953 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:26.003764 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:26.003953 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:26.003859 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:26.004366 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:26.004002 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:27.003116 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:27.003084 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:27.003334 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.003190 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:27.645558 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:27.645519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:27.645917 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.645681 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:27.645917 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.645746 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 20:59:43.645730626 +0000 UTC m=+34.636277127 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:27.746861 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:27.746826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:27.747056 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.746957 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:27.747056 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.747025 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:43.747006155 +0000 UTC m=+34.737552662 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:27.847337 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:27.847294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:27.847500 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.847447 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:27.847500 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.847471 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:27.847500 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.847481 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:27.847627 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:27.847534 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 20:59:43.847519517 +0000 UTC m=+34.838066018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:28.003372 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:28.003282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:28.003567 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:28.003282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:28.003567 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:28.003431 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:28.003567 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:28.003506 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:29.003500 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:29.003435 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:29.003960 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:29.003613 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:30.004056 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:30.004016 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:30.004520 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:30.004153 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:30.004520 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:30.004206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:30.004520 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:30.004325 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:31.003022 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:31.002980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:31.003179 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:31.003103 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:32.003303 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.003130 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:32.004103 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.003129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:32.004103 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:32.003415 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:32.004103 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:32.003451 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:32.101322 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.101293 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-stjx7" event={"ID":"0213c346-df99-4945-9f79-884622623238","Type":"ContainerStarted","Data":"821373f8c01a35deee370122a000515c4efc1402599b9a81c31016c18bc9362e"} Jul 23 20:59:32.103209 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.102974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d8j56" event={"ID":"b22b0230-000b-4fd7-8596-b5832724cecb","Type":"ContainerStarted","Data":"fc8f19107bcba00f8ed815135edc7414592a59aee5626f99b0869545d5bcaa90"} Jul 23 20:59:32.105064 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.105004 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-72tl8" event={"ID":"2b959712-8ea5-4d5c-97c8-c53de4c55ad9","Type":"ContainerStarted","Data":"cf05b8215769de29e3f4d2203e6afacd8ef60f66adfb0942572cfdd5ea0b18f9"} Jul 23 20:59:32.109427 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.109397 2578 generic.go:356] "Generic (PLEG): container finished" podID="549e7963-6203-406f-9c29-2716303002d5" containerID="83caa1c7a4ab737c59ad77aed3806f22e012857ff909666bbb947b98296708d3" exitCode=0 Jul 23 20:59:32.109573 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.109472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerDied","Data":"83caa1c7a4ab737c59ad77aed3806f22e012857ff909666bbb947b98296708d3"} Jul 23 20:59:32.111045 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.111023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" event={"ID":"92ec2b16-a339-487b-9f50-8f58b9e1825b","Type":"ContainerStarted","Data":"605341cb9425756416b22bddde7fce86274dd4a59b8dc7c0d49f76ec9947c17f"} Jul 23 20:59:32.112995 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.112968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"04d00833521a579ecbaeb810c5af110d88f363222f83eb8f3d59d0ce3f8aeda0"} Jul 23 20:59:32.113078 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.113031 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"e52cfbe2a700814820b87e754363a3fcafc18411fd130cb56b712c6d0afe52ab"} Jul 23 20:59:32.113078 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.113063 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"87ca815568a872d33c444560b257dc09e75205c14737b68bbcdef4b4cb5ab5f0"} Jul 23 20:59:32.114245 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.114215 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" event={"ID":"fe72a6e4-1476-406c-85c0-798df5aab117","Type":"ContainerStarted","Data":"7a122d1808ced6b83fd157a4599e3d780d1b64eff5f17450b8a39b1388dd25ca"} Jul 23 20:59:32.115468 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.115449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dqtbt" event={"ID":"2306fe4d-4766-4135-834e-8f2d7e916321","Type":"ContainerStarted","Data":"0e63f13dceb3486c5a961b775b5fb9747c9cc564cb488d3c54fc84fd05dd4e1e"} Jul 23 20:59:32.118824 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.118782 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-stjx7" podStartSLOduration=7.325744448 podStartE2EDuration="22.118771744s" totalImagesPullingTime="14.793027296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:32.117814219 +0000 UTC m=+23.108360741" watchObservedRunningTime="2026-07-23 20:59:32.118771744 +0000 UTC m=+23.109318265" Jul 23 20:59:32.166860 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.166801 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-72tl8" podStartSLOduration=3.396532968 podStartE2EDuration="22.166779297s" totalImagesPullingTime="18.770246329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:32.152764301 +0000 UTC m=+23.143310824" watchObservedRunningTime="2026-07-23 20:59:32.166779297 +0000 UTC m=+23.157325822" Jul 23 20:59:32.167520 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.167477 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d8j56" podStartSLOduration=3.398822348 podStartE2EDuration="22.167465205s" totalImagesPullingTime="18.768642857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:32.166605061 +0000 UTC m=+23.157151603" watchObservedRunningTime="2026-07-23 20:59:32.167465205 +0000 UTC m=+23.158011731" Jul 23 20:59:32.183831 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.183718 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dqtbt" podStartSLOduration=3.338465007 podStartE2EDuration="22.183701337s" totalImagesPullingTime="18.84523633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:32.183284691 +0000 UTC m=+23.173831215" watchObservedRunningTime="2026-07-23 20:59:32.183701337 +0000 UTC m=+23.174247890" Jul 23 20:59:32.201006 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:32.200927 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hlqpk" podStartSLOduration=3.423965225 podStartE2EDuration="22.200901312s" totalImagesPullingTime="18.776936087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:32.200779975 +0000 UTC m=+23.191326498" watchObservedRunningTime="2026-07-23 20:59:32.200901312 +0000 UTC m=+23.191447834" Jul 23 20:59:33.003599 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.003501 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:33.004105 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:33.003644 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:33.121437 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.121399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"f2b1c9743be0be41cd0fb20d4f672246edcb2f5bd14907f5e6b8e772c170fa26"} Jul 23 20:59:33.121655 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.121494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"377b1a4a20928e25c46e1f307b820eec2c0401eda44bdb30f15812760404ed62"} Jul 23 20:59:33.121655 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.121559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"7cc8d777447fb5481b7bea4441dedb43d8dbdb4a48da8ec8af49bd1ebc0c2ad5"} Jul 23 20:59:33.167497 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.167465 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 20:59:33.979016 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.978885 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T20:59:33.16749116Z","UUID":"805b49fd-c6ff-48a5-bf6d-d4cab4c735bf","Handler":null,"Name":"","Endpoint":""} Jul 23 20:59:33.980880 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.980785 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 20:59:33.980880 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:33.980824 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 20:59:34.003328 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:34.003294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:34.003513 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:34.003438 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:34.003738 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:34.003714 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:34.004170 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:34.003821 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:34.125373 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:34.125331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" event={"ID":"fe72a6e4-1476-406c-85c0-798df5aab117","Type":"ContainerStarted","Data":"a1c03af50d9a578e2e54e0cffc56d0680d0aabe8ce77b928e3124539c59b0de4"} Jul 23 20:59:34.127112 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:34.127080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k82w9" event={"ID":"df6d5abf-4234-467e-97c5-996ca00b98e3","Type":"ContainerStarted","Data":"e46fbe42a81341c301eff076a2be3a1fbaeaf26aaa0193ac4aaa3b0503d2ec6a"} Jul 23 20:59:34.140870 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:34.140816 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k82w9" podStartSLOduration=5.36938883 podStartE2EDuration="24.140798297s" totalImagesPullingTime="18.771409467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:34.140526608 +0000 UTC m=+25.131073132" watchObservedRunningTime="2026-07-23 20:59:34.140798297 +0000 UTC m=+25.131344820" Jul 23 20:59:35.003826 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:35.003792 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:35.004422 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:35.003920 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:35.134095 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:35.134057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"681368d247065dd35df9d1ecbbcfe57351f01ff4680285d4b83815fd40e7a17a"} Jul 23 20:59:35.136432 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:35.136404 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" event={"ID":"fe72a6e4-1476-406c-85c0-798df5aab117","Type":"ContainerStarted","Data":"1196d95b86bb16f99a8f9c4f879ed8d75a1eb47f7e2fd1b5b0318064624c45ba"} Jul 23 20:59:35.154612 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:35.154540 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdnrt" podStartSLOduration=4.105445265 podStartE2EDuration="25.154523956s" totalImagesPullingTime="21.049078691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:35.154487994 +0000 UTC m=+26.145034539" watchObservedRunningTime="2026-07-23 20:59:35.154523956 +0000 UTC m=+26.145070475" Jul 23 20:59:35.909711 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:35.909672 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:35.910385 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:35.910364 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:36.003747 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:36.003714 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:36.003928 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:36.003714 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:36.003928 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:36.003855 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:36.004324 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:36.003924 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:37.003948 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.003764 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:37.004720 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:37.004044 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:37.143044 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.142958 2578 generic.go:356] "Generic (PLEG): container finished" podID="549e7963-6203-406f-9c29-2716303002d5" containerID="49352ace91e5131dad35dc304daa46a317dcc3fae0506337b119865755a49522" exitCode=0 Jul 23 20:59:37.143044 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.143019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerDied","Data":"49352ace91e5131dad35dc304daa46a317dcc3fae0506337b119865755a49522"} Jul 23 20:59:37.146992 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.146964 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" event={"ID":"a600fc50-247e-4a0d-8e7d-a7f3e5396cd1","Type":"ContainerStarted","Data":"8379e8f93a1514d9d19bb4a246427f6de603ad52824a6c6bdbf9439bd3ec7aa1"} Jul 23 20:59:37.147226 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.147186 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:37.147358 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.147249 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:37.147358 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.147297 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:37.163343 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.163315 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:37.167050 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.167023 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 20:59:37.188030 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:37.187976 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" podStartSLOduration=8.351666759 podStartE2EDuration="27.187960268s" totalImagesPullingTime="18.836293509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:37.18762417 +0000 UTC m=+28.178170714" watchObservedRunningTime="2026-07-23 20:59:37.187960268 +0000 UTC m=+28.178506791" Jul 23 20:59:38.003737 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.003536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:38.003977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.003536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:38.003977 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:38.003819 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:38.003977 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:38.003950 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:38.826570 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.826515 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bs7mm"] Jul 23 20:59:38.826884 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.826715 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:38.826884 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:38.826812 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:38.827239 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.827218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vcsvx"] Jul 23 20:59:38.827338 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.827325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:38.827438 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:38.827419 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:38.832460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.832433 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6r9p8"] Jul 23 20:59:38.832601 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:38.832562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:38.832698 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:38.832678 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:39.153000 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:39.152899 2578 generic.go:356] "Generic (PLEG): container finished" podID="549e7963-6203-406f-9c29-2716303002d5" containerID="484ffa8c64cd778614c1a5ec9c89c979daa5e5ec177663cc2196b73cc86e1c5f" exitCode=0 Jul 23 20:59:39.153000 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:39.152991 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerDied","Data":"484ffa8c64cd778614c1a5ec9c89c979daa5e5ec177663cc2196b73cc86e1c5f"} Jul 23 20:59:39.979038 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:39.979009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:39.979201 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:39.979135 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 20:59:39.979539 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:39.979526 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-stjx7" Jul 23 20:59:40.006523 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:40.006497 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:40.006685 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:40.006603 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:41.003408 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:41.003317 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:41.003860 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:41.003317 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:41.003860 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:41.003435 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:41.003860 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:41.003539 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:41.159365 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:41.159329 2578 generic.go:356] "Generic (PLEG): container finished" podID="549e7963-6203-406f-9c29-2716303002d5" containerID="3c9a45c6fa47436fc86539c913a90bac75f01b8a895ef56b0bc9195523ad44d7" exitCode=0 Jul 23 20:59:41.159521 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:41.159390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerDied","Data":"3c9a45c6fa47436fc86539c913a90bac75f01b8a895ef56b0bc9195523ad44d7"} Jul 23 20:59:42.003098 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:42.002863 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:42.003290 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:42.003200 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:43.003454 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:43.003412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:43.003912 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.003556 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bs7mm" podUID="a6a4f6f0-c1d5-462f-ab87-eefaa2943c56" Jul 23 20:59:43.003912 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:43.003611 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:43.003912 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.003739 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r9p8" podUID="f3229463-d919-4499-9c87-61ab3de151ff" Jul 23 20:59:43.671103 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:43.671063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:43.671288 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.671229 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:43.671359 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.671289 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs podName:f3229463-d919-4499-9c87-61ab3de151ff nodeName:}" failed. No retries permitted until 2026-07-23 21:00:15.671273834 +0000 UTC m=+66.661820334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs") pod "network-metrics-daemon-6r9p8" (UID: "f3229463-d919-4499-9c87-61ab3de151ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 20:59:43.771494 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:43.771458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:43.771694 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.771611 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:43.771694 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.771685 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret podName:a6a4f6f0-c1d5-462f-ab87-eefaa2943c56 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:15.771662894 +0000 UTC m=+66.762209407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret") pod "global-pull-secret-syncer-bs7mm" (UID: "a6a4f6f0-c1d5-462f-ab87-eefaa2943c56") : object "kube-system"/"original-pull-secret" not registered Jul 23 20:59:43.872490 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:43.872450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:43.872676 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.872628 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 20:59:43.872676 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.872645 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 20:59:43.872676 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.872655 2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7gf5 for pod openshift-network-diagnostics/network-check-target-vcsvx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:43.872809 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:43.872714 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5 podName:da30a5a0-f5ab-4000-9c29-c7c829d7b7ab nodeName:}" failed. No retries permitted until 2026-07-23 21:00:15.872696977 +0000 UTC m=+66.863243479 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7gf5" (UniqueName: "kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5") pod "network-check-target-vcsvx" (UID: "da30a5a0-f5ab-4000-9c29-c7c829d7b7ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 20:59:44.003060 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.002980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:44.003210 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.003122 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vcsvx" podUID="da30a5a0-f5ab-4000-9c29-c7c829d7b7ab" Jul 23 20:59:44.278989 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.278915 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-43.ec2.internal" event="NodeReady" Jul 23 20:59:44.279412 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.279055 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 20:59:44.313642 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.313610 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t"] Jul 23 20:59:44.332809 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.332767 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8g727"] Jul 23 20:59:44.332987 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.332912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.335643 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.335610 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.335899 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.335877 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gc9km\"" Jul 23 20:59:44.336310 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.336280 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 20:59:44.336405 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.336283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.336405 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.336349 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 20:59:44.345485 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.345456 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d54bbc4cb-htvqd"] Jul 23 20:59:44.345638 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.345619 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.349533 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.349509 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 20:59:44.350035 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.350013 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 20:59:44.350241 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.350064 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-np8jf\"" Jul 23 20:59:44.350378 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.350313 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.352983 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.352949 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.359492 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.359213 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 20:59:44.361903 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.361192 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-96c8965f-8wk9l"] Jul 23 20:59:44.362596 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.362191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.365194 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.364500 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 20:59:44.365194 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.364868 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7xbv6\"" Jul 23 20:59:44.369745 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.369684 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.370149 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.370125 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.370274 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.370242 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 20:59:44.370386 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.370358 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 20:59:44.370659 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.370638 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 20:59:44.373338 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.373304 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t"] Jul 23 20:59:44.373493 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.373346 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn"] Jul 23 20:59:44.373493 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.373443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.377330 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.377304 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 20:59:44.377681 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.377660 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 20:59:44.377997 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.377979 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-km8vb\"" Jul 23 20:59:44.378281 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.378076 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 20:59:44.379782 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.379496 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rwvmk"] Jul 23 20:59:44.379782 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.379711 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" Jul 23 20:59:44.384081 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.383452 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bsrqz\"" Jul 23 20:59:44.384081 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.383897 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.384360 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.384122 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.388724 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.388701 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 20:59:44.391386 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.391359 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4"] Jul 23 20:59:44.391527 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.391499 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.394675 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.394343 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 20:59:44.394675 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.394413 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 20:59:44.394675 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.394437 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.394675 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.394437 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-sn7jm\"" Jul 23 20:59:44.394675 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.394628 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.400331 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.399032 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9"] Jul 23 20:59:44.400331 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.399183 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:44.400331 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.400083 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 20:59:44.402089 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.402070 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.402668 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.402645 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 23 20:59:44.402792 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.402772 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zkktg\"" Jul 23 20:59:44.402908 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.402844 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.409510 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.409486 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj"] Jul 23 20:59:44.409686 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.409670 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:44.412257 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.412223 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 20:59:44.412257 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.412254 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bv2bf\"" Jul 23 20:59:44.412431 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.412224 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 20:59:44.417426 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.417394 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d54bbc4cb-htvqd"] Jul 23 20:59:44.417567 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.417443 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx"] Jul 23 20:59:44.417634 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.417607 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" Jul 23 20:59:44.420313 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.420290 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.420441 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.420422 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.420877 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.420846 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-x6kxj\"" Jul 23 20:59:44.425647 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.425621 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l"] Jul 23 20:59:44.425855 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.425783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.427967 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.427947 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 20:59:44.428246 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.428225 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.428334 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.428253 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.428334 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.428302 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6sjmr\"" Jul 23 20:59:44.428523 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.428496 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 20:59:44.433954 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.433930 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96c8965f-8wk9l"] Jul 23 20:59:44.434062 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.433970 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn"] Jul 23 20:59:44.434062 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.433992 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8g727"] Jul 23 20:59:44.434062 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.434010 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rwvmk"] Jul 23 20:59:44.434062 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.434023 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4"] Jul 23 20:59:44.434062 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.434041 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj"] Jul 23 20:59:44.434062 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.434057 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ffvc4"] Jul 23 20:59:44.434352 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.434083 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.436344 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.436317 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 20:59:44.436344 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.436331 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.436344 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.436337 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 20:59:44.436671 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.436370 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qf8v6\"" Jul 23 20:59:44.436671 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.436411 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.449945 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.449908 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pmf6p"] Jul 23 20:59:44.450257 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.450240 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.452602 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.452579 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 20:59:44.452713 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.452654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 20:59:44.452713 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.452657 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 20:59:44.452713 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.452681 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k59g8\"" Jul 23 20:59:44.466083 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.466045 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9"] Jul 23 20:59:44.466240 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.466101 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx"] Jul 23 20:59:44.466240 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.466118 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l"] Jul 23 20:59:44.466240 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.466132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmf6p"] Jul 23 20:59:44.466240 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.466149 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ffvc4"] Jul 23 20:59:44.466240 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.466189 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.469083 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.469059 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 20:59:44.469427 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.469400 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dj8rg\"" Jul 23 20:59:44.469645 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.469619 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 20:59:44.478744 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/66d92b23-a2bf-442c-9986-0a10dcd10d7e-serving-cert\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.478901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62csc\" (UniqueName: \"kubernetes.io/projected/8f41b4bf-7378-43f8-9f55-a63019753a5e-kube-api-access-62csc\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:44.478901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/43b49123-f63a-45cf-a5e9-12aa4f0eff24-config\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.478901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-image-registry-private-configuration\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.478901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-trusted-ca\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.478901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478918 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-bound-sa-token\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8784\" (UniqueName: \"kubernetes.io/projected/66d92b23-a2bf-442c-9986-0a10dcd10d7e-kube-api-access-d8784\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.478992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54mx\" (UniqueName: \"kubernetes.io/projected/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-kube-api-access-p54mx\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/43b49123-f63a-45cf-a5e9-12aa4f0eff24-serving-cert\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.479115 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66d92b23-a2bf-442c-9986-0a10dcd10d7e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-default-certificate\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbnnk\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-kube-api-access-mbnnk\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a886bb36-fb28-4d64-95a7-fbeb2873542a-ca-trust-extracted\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479245 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-stats-auth\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66lbl\" (UniqueName: \"kubernetes.io/projected/3114df95-e113-40a9-a92e-dbf21f91ff1b-kube-api-access-66lbl\") pod \"volume-data-source-validator-78cf4c6f6-sx5mn\" (UID: \"3114df95-e113-40a9-a92e-dbf21f91ff1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" Jul 23 20:59:44.479361 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/66d92b23-a2bf-442c-9986-0a10dcd10d7e-snapshots\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c6f338b8-691d-4985-9398-0452739f7aec-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-installation-pull-secrets\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b49123-f63a-45cf-a5e9-12aa4f0eff24-trusted-ca\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-certificates\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66d92b23-a2bf-442c-9986-0a10dcd10d7e-tmp\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479560 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66d92b23-a2bf-442c-9986-0a10dcd10d7e-service-ca-bundle\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.479670 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb765\" (UniqueName: \"kubernetes.io/projected/43b49123-f63a-45cf-a5e9-12aa4f0eff24-kube-api-access-mb765\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.479886 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.479697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz2m9\" (UniqueName: \"kubernetes.io/projected/c6f338b8-691d-4985-9398-0452739f7aec-kube-api-access-pz2m9\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.580327 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.580327 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.580572 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54mx\" (UniqueName: \"kubernetes.io/projected/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-kube-api-access-p54mx\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.580572 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.580399 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.080377181 +0000 UTC m=+36.070923681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : configmap references non-existent config key: service-ca.crt Jul 23 20:59:44.580572 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580434 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqhzp\" (UniqueName: \"kubernetes.io/projected/ccf68472-95b3-4bf4-acf8-25ecb1726591-kube-api-access-xqhzp\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.580572 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.580461 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:44.580572 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.580537 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls podName:c6f338b8-691d-4985-9398-0452739f7aec nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.080517809 +0000 UTC m=+36.071064315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tbs7t" (UID: "c6f338b8-691d-4985-9398-0452739f7aec") : secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:44.580839 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/43b49123-f63a-45cf-a5e9-12aa4f0eff24-serving-cert\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.580839 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66d92b23-a2bf-442c-9986-0a10dcd10d7e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.580839 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj9hj\" (UniqueName: \"kubernetes.io/projected/e724dd31-c672-47de-b929-30bb27fe25ef-kube-api-access-rj9hj\") pod \"ingress-canary-ffvc4\" (UID: \"e724dd31-c672-47de-b929-30bb27fe25ef\") " pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.580839 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d06fea94-cf37-4c37-b739-350370a4699c-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:44.580839 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-default-certificate\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbnnk\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-kube-api-access-mbnnk\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a886bb36-fb28-4d64-95a7-fbeb2873542a-ca-trust-extracted\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-config\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.580994 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e724dd31-c672-47de-b929-30bb27fe25ef-cert\") pod \"ingress-canary-ffvc4\" (UID: \"e724dd31-c672-47de-b929-30bb27fe25ef\") " pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-stats-auth\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.581085 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66lbl\" (UniqueName: \"kubernetes.io/projected/3114df95-e113-40a9-a92e-dbf21f91ff1b-kube-api-access-66lbl\") pod \"volume-data-source-validator-78cf4c6f6-sx5mn\" (UID: \"3114df95-e113-40a9-a92e-dbf21f91ff1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5wtr\" (UniqueName: \"kubernetes.io/projected/9fd59ff0-d9ae-4220-b427-0a17ed800988-kube-api-access-f5wtr\") pod \"network-check-source-7bc5b6bf85-zm8xj\" (UID: \"9fd59ff0-d9ae-4220-b427-0a17ed800988\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.581130 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/66d92b23-a2bf-442c-9986-0a10dcd10d7e-snapshots\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.581205 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.081175816 +0000 UTC m=+36.071722319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : secret "router-metrics-certs-default" not found Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.581260 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.581274 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96c8965f-8wk9l: secret "image-registry-tls" not found Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c6f338b8-691d-4985-9398-0452739f7aec-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a886bb36-fb28-4d64-95a7-fbeb2873542a-ca-trust-extracted\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.581326 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls podName:a886bb36-fb28-4d64-95a7-fbeb2873542a nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.0813147 +0000 UTC m=+36.071861200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls") pod "image-registry-96c8965f-8wk9l" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a") : secret "image-registry-tls" not found Jul 23 20:59:44.581460 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-installation-pull-secrets\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.582063 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581856 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66d92b23-a2bf-442c-9986-0a10dcd10d7e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.582063 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581874 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/66d92b23-a2bf-442c-9986-0a10dcd10d7e-snapshots\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.582063 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.581915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b49123-f63a-45cf-a5e9-12aa4f0eff24-trusted-ca\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.582063 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582008 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c6f338b8-691d-4985-9398-0452739f7aec-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582141 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-certificates\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66d92b23-a2bf-442c-9986-0a10dcd10d7e-tmp\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66d92b23-a2bf-442c-9986-0a10dcd10d7e-service-ca-bundle\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb765\" (UniqueName: \"kubernetes.io/projected/43b49123-f63a-45cf-a5e9-12aa4f0eff24-kube-api-access-mb765\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz2m9\" (UniqueName: \"kubernetes.io/projected/c6f338b8-691d-4985-9398-0452739f7aec-kube-api-access-pz2m9\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/66d92b23-a2bf-442c-9986-0a10dcd10d7e-serving-cert\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ccf68472-95b3-4bf4-acf8-25ecb1726591-config-volume\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582366 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62csc\" (UniqueName: \"kubernetes.io/projected/8f41b4bf-7378-43f8-9f55-a63019753a5e-kube-api-access-62csc\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/43b49123-f63a-45cf-a5e9-12aa4f0eff24-config\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-image-registry-private-configuration\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-trusted-ca\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccf68472-95b3-4bf4-acf8-25ecb1726591-tmp-dir\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:44.583915 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zprc6\" (UniqueName: \"kubernetes.io/projected/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-kube-api-access-zprc6\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-bound-sa-token\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8784\" (UniqueName: \"kubernetes.io/projected/66d92b23-a2bf-442c-9986-0a10dcd10d7e-kube-api-access-d8784\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582674 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltcjf\" (UniqueName: \"kubernetes.io/projected/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-kube-api-access-ltcjf\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582729 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-certificates\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582795 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43b49123-f63a-45cf-a5e9-12aa4f0eff24-trusted-ca\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.582806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66d92b23-a2bf-442c-9986-0a10dcd10d7e-tmp\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.583153 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.583212 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls podName:8f41b4bf-7378-43f8-9f55-a63019753a5e nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.083195774 +0000 UTC m=+36.073742287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-srch4" (UID: "8f41b4bf-7378-43f8-9f55-a63019753a5e") : secret "samples-operator-tls" not found Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.583445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/43b49123-f63a-45cf-a5e9-12aa4f0eff24-config\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.583667 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-trusted-ca\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.585208 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.583879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66d92b23-a2bf-442c-9986-0a10dcd10d7e-service-ca-bundle\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.586977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.586932 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/43b49123-f63a-45cf-a5e9-12aa4f0eff24-serving-cert\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.586977 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.586970 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/66d92b23-a2bf-442c-9986-0a10dcd10d7e-serving-cert\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.587142 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.587049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-installation-pull-secrets\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.587142 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.587071 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-default-certificate\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.587142 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.587071 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-stats-auth\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.587330 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.587303 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-image-registry-private-configuration\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.592316 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.592289 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbnnk\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-kube-api-access-mbnnk\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.593096 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.593073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66lbl\" (UniqueName: \"kubernetes.io/projected/3114df95-e113-40a9-a92e-dbf21f91ff1b-kube-api-access-66lbl\") pod \"volume-data-source-validator-78cf4c6f6-sx5mn\" (UID: \"3114df95-e113-40a9-a92e-dbf21f91ff1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" Jul 23 20:59:44.596492 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.596439 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz2m9\" (UniqueName: \"kubernetes.io/projected/c6f338b8-691d-4985-9398-0452739f7aec-kube-api-access-pz2m9\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:44.597092 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.597065 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-bound-sa-token\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:44.597209 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.597092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb765\" (UniqueName: \"kubernetes.io/projected/43b49123-f63a-45cf-a5e9-12aa4f0eff24-kube-api-access-mb765\") pod \"console-operator-565b7b569d-rwvmk\" (UID: \"43b49123-f63a-45cf-a5e9-12aa4f0eff24\") " pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.597508 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.597445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62csc\" (UniqueName: \"kubernetes.io/projected/8f41b4bf-7378-43f8-9f55-a63019753a5e-kube-api-access-62csc\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:44.598693 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.598670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8784\" (UniqueName: \"kubernetes.io/projected/66d92b23-a2bf-442c-9986-0a10dcd10d7e-kube-api-access-d8784\") pod \"insights-operator-555d9d7885-8g727\" (UID: \"66d92b23-a2bf-442c-9986-0a10dcd10d7e\") " pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.601844 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.601824 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54mx\" (UniqueName: \"kubernetes.io/projected/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-kube-api-access-p54mx\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:44.658648 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.658606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8g727" Jul 23 20:59:44.683665 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccf68472-95b3-4bf4-acf8-25ecb1726591-tmp-dir\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zprc6\" (UniqueName: \"kubernetes.io/projected/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-kube-api-access-zprc6\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltcjf\" (UniqueName: \"kubernetes.io/projected/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-kube-api-access-ltcjf\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqhzp\" (UniqueName: \"kubernetes.io/projected/ccf68472-95b3-4bf4-acf8-25ecb1726591-kube-api-access-xqhzp\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj9hj\" (UniqueName: \"kubernetes.io/projected/e724dd31-c672-47de-b929-30bb27fe25ef-kube-api-access-rj9hj\") pod \"ingress-canary-ffvc4\" (UID: \"e724dd31-c672-47de-b929-30bb27fe25ef\") " pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.683859 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d06fea94-cf37-4c37-b739-350370a4699c-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-config\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683954 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e724dd31-c672-47de-b929-30bb27fe25ef-cert\") pod \"ingress-canary-ffvc4\" (UID: \"e724dd31-c672-47de-b929-30bb27fe25ef\") " pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.683987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5wtr\" (UniqueName: \"kubernetes.io/projected/9fd59ff0-d9ae-4220-b427-0a17ed800988-kube-api-access-f5wtr\") pod \"network-check-source-7bc5b6bf85-zm8xj\" (UID: \"9fd59ff0-d9ae-4220-b427-0a17ed800988\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.684018 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.684027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccf68472-95b3-4bf4-acf8-25ecb1726591-tmp-dir\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.684061 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.684130 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.684189 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.684158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ccf68472-95b3-4bf4-acf8-25ecb1726591-config-volume\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.684658 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.684272 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 20:59:44.684658 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.684331 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls podName:ccf68472-95b3-4bf4-acf8-25ecb1726591 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.184312095 +0000 UTC m=+36.174858603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls") pod "dns-default-pmf6p" (UID: "ccf68472-95b3-4bf4-acf8-25ecb1726591") : secret "dns-default-metrics-tls" not found Jul 23 20:59:44.684658 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.684461 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 20:59:44.684658 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:44.684503 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert podName:d06fea94-cf37-4c37-b739-350370a4699c nodeName:}" failed. No retries permitted until 2026-07-23 20:59:45.184488843 +0000 UTC m=+36.175035359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9tsh9" (UID: "d06fea94-cf37-4c37-b739-350370a4699c") : secret "networking-console-plugin-cert" not found Jul 23 20:59:44.684874 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.684778 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ccf68472-95b3-4bf4-acf8-25ecb1726591-config-volume\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.685070 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.685026 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.685177 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.685102 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d06fea94-cf37-4c37-b739-350370a4699c-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:44.685177 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.685108 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-config\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.686941 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.686898 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.687518 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.687497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e724dd31-c672-47de-b929-30bb27fe25ef-cert\") pod \"ingress-canary-ffvc4\" (UID: \"e724dd31-c672-47de-b929-30bb27fe25ef\") " pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.687917 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.687895 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.693353 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.693314 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zprc6\" (UniqueName: \"kubernetes.io/projected/2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c-kube-api-access-zprc6\") pod \"service-ca-operator-5bcbc94c78-h4d5l\" (UID: \"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.693699 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.693653 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5wtr\" (UniqueName: \"kubernetes.io/projected/9fd59ff0-d9ae-4220-b427-0a17ed800988-kube-api-access-f5wtr\") pod \"network-check-source-7bc5b6bf85-zm8xj\" (UID: \"9fd59ff0-d9ae-4220-b427-0a17ed800988\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" Jul 23 20:59:44.694007 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.693982 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqhzp\" (UniqueName: \"kubernetes.io/projected/ccf68472-95b3-4bf4-acf8-25ecb1726591-kube-api-access-xqhzp\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:44.694108 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.694069 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj9hj\" (UniqueName: \"kubernetes.io/projected/e724dd31-c672-47de-b929-30bb27fe25ef-kube-api-access-rj9hj\") pod \"ingress-canary-ffvc4\" (UID: \"e724dd31-c672-47de-b929-30bb27fe25ef\") " pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:44.694260 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.694207 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltcjf\" (UniqueName: \"kubernetes.io/projected/2565a914-16d9-4bf9-8adb-f07b5bcaaff4-kube-api-access-ltcjf\") pod \"kube-storage-version-migrator-operator-7658d9b749-4b6xx\" (UID: \"2565a914-16d9-4bf9-8adb-f07b5bcaaff4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.698083 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.698049 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" Jul 23 20:59:44.707496 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.707455 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:44.728898 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.728865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" Jul 23 20:59:44.737197 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.737166 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" Jul 23 20:59:44.745325 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.745297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" Jul 23 20:59:44.761398 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:44.761362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ffvc4" Jul 23 20:59:45.003954 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.003866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 20:59:45.003954 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.003919 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 20:59:45.006601 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.006572 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 20:59:45.006601 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.006595 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 20:59:45.006814 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.006595 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-btkpz\"" Jul 23 20:59:45.088458 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.088424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:45.088458 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.088470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.088607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088630 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.088651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.088675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088688 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088702 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.088681864 +0000 UTC m=+37.079228387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : secret "router-metrics-certs-default" not found Jul 23 20:59:45.088730 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088640 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 20:59:45.089016 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088738 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96c8965f-8wk9l: secret "image-registry-tls" not found Jul 23 20:59:45.089016 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088739 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls podName:8f41b4bf-7378-43f8-9f55-a63019753a5e nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.088727447 +0000 UTC m=+37.079273954 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-srch4" (UID: "8f41b4bf-7378-43f8-9f55-a63019753a5e") : secret "samples-operator-tls" not found Jul 23 20:59:45.089016 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088772 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.08875394 +0000 UTC m=+37.079300441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : configmap references non-existent config key: service-ca.crt Jul 23 20:59:45.089016 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088799 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls podName:a886bb36-fb28-4d64-95a7-fbeb2873542a nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.08878759 +0000 UTC m=+37.079334096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls") pod "image-registry-96c8965f-8wk9l" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a") : secret "image-registry-tls" not found Jul 23 20:59:45.089016 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088804 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:45.089016 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.088837 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls podName:c6f338b8-691d-4985-9398-0452739f7aec nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.08882691 +0000 UTC m=+37.079373420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tbs7t" (UID: "c6f338b8-691d-4985-9398-0452739f7aec") : secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:45.190042 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.190002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:45.190237 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.190192 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 20:59:45.190305 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.190276 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert podName:d06fea94-cf37-4c37-b739-350370a4699c nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.190255436 +0000 UTC m=+37.180801955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9tsh9" (UID: "d06fea94-cf37-4c37-b739-350370a4699c") : secret "networking-console-plugin-cert" not found Jul 23 20:59:45.190383 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.190365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:45.190492 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.190478 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 20:59:45.190563 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:45.190519 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls podName:ccf68472-95b3-4bf4-acf8-25ecb1726591 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:46.190509658 +0000 UTC m=+37.181056160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls") pod "dns-default-pmf6p" (UID: "ccf68472-95b3-4bf4-acf8-25ecb1726591") : secret "dns-default-metrics-tls" not found Jul 23 20:59:45.541101 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.540801 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l"] Jul 23 20:59:45.546227 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.546178 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd2a3fa_d46b_4bb6_bd36_b5a1a9922a3c.slice/crio-4383fe0c56e3463214b547275fcbca92fbd209901f6bdfed1a2f37586973c88d WatchSource:0}: Error finding container 4383fe0c56e3463214b547275fcbca92fbd209901f6bdfed1a2f37586973c88d: Status 404 returned error can't find the container with id 4383fe0c56e3463214b547275fcbca92fbd209901f6bdfed1a2f37586973c88d Jul 23 20:59:45.551442 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.551373 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn"] Jul 23 20:59:45.555318 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.555295 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx"] Jul 23 20:59:45.558256 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.558232 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3114df95_e113_40a9_a92e_dbf21f91ff1b.slice/crio-32cec1f02a9834ef2f2ed2f46687257e0d114f17c79850adffbb6ee58a2afc22 WatchSource:0}: Error finding container 32cec1f02a9834ef2f2ed2f46687257e0d114f17c79850adffbb6ee58a2afc22: Status 404 returned error can't find the container with id 32cec1f02a9834ef2f2ed2f46687257e0d114f17c79850adffbb6ee58a2afc22 Jul 23 20:59:45.559084 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.559062 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2565a914_16d9_4bf9_8adb_f07b5bcaaff4.slice/crio-720a9feaa4634db1ab8870a3d41cecabec480b7f4596944a1f63c99456a6f9cb WatchSource:0}: Error finding container 720a9feaa4634db1ab8870a3d41cecabec480b7f4596944a1f63c99456a6f9cb: Status 404 returned error can't find the container with id 720a9feaa4634db1ab8870a3d41cecabec480b7f4596944a1f63c99456a6f9cb Jul 23 20:59:45.562900 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.562868 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj"] Jul 23 20:59:45.565500 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.565444 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rwvmk"] Jul 23 20:59:45.566323 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.566301 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd59ff0_d9ae_4220_b427_0a17ed800988.slice/crio-d6504fb87987d5142d1e9b1a5c11b91e377e4157c0c164a6cf828dd1f23fe78a WatchSource:0}: Error finding container d6504fb87987d5142d1e9b1a5c11b91e377e4157c0c164a6cf828dd1f23fe78a: Status 404 returned error can't find the container with id d6504fb87987d5142d1e9b1a5c11b91e377e4157c0c164a6cf828dd1f23fe78a Jul 23 20:59:45.568336 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.568255 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ffvc4"] Jul 23 20:59:45.570953 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.570760 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b49123_f63a_45cf_a5e9_12aa4f0eff24.slice/crio-32a8272320d1002597b9d2795548d37d0e5f411f074736bf8ca908cfbc4edce6 WatchSource:0}: Error finding container 32a8272320d1002597b9d2795548d37d0e5f411f074736bf8ca908cfbc4edce6: Status 404 returned error can't find the container with id 32a8272320d1002597b9d2795548d37d0e5f411f074736bf8ca908cfbc4edce6 Jul 23 20:59:45.575250 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.575230 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode724dd31_c672_47de_b929_30bb27fe25ef.slice/crio-a7f30a05abe400f1eae259df875cc774859e721dea34d0389d01a0788c3a27c5 WatchSource:0}: Error finding container a7f30a05abe400f1eae259df875cc774859e721dea34d0389d01a0788c3a27c5: Status 404 returned error can't find the container with id a7f30a05abe400f1eae259df875cc774859e721dea34d0389d01a0788c3a27c5 Jul 23 20:59:45.581372 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:45.581195 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8g727"] Jul 23 20:59:45.583802 ip-10-0-129-43 kubenswrapper[2578]: W0723 20:59:45.583771 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d92b23_a2bf_442c_9986_0a10dcd10d7e.slice/crio-de1f7c62061bf51cb0fcd023c5fc3f15733ff6b0db9be4be87e8b0f0da31e833 WatchSource:0}: Error finding container de1f7c62061bf51cb0fcd023c5fc3f15733ff6b0db9be4be87e8b0f0da31e833: Status 404 returned error can't find the container with id de1f7c62061bf51cb0fcd023c5fc3f15733ff6b0db9be4be87e8b0f0da31e833 Jul 23 20:59:46.003889 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.003809 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 20:59:46.006834 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.006806 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9qkcv\"" Jul 23 20:59:46.102713 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.102666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:46.102901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.102734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:46.102901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.102766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:46.102901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.102819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:46.102901 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.102850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:46.103102 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103056 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 20:59:46.103102 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103071 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96c8965f-8wk9l: secret "image-registry-tls" not found Jul 23 20:59:46.103202 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103131 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls podName:a886bb36-fb28-4d64-95a7-fbeb2873542a nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.103112757 +0000 UTC m=+39.093659260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls") pod "image-registry-96c8965f-8wk9l" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a") : secret "image-registry-tls" not found Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103559 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103612 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls podName:8f41b4bf-7378-43f8-9f55-a63019753a5e nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.103597441 +0000 UTC m=+39.094143942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-srch4" (UID: "8f41b4bf-7378-43f8-9f55-a63019753a5e") : secret "samples-operator-tls" not found Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103671 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103702 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls podName:c6f338b8-691d-4985-9398-0452739f7aec nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.103692603 +0000 UTC m=+39.094239109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tbs7t" (UID: "c6f338b8-691d-4985-9398-0452739f7aec") : secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103764 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.103754756 +0000 UTC m=+39.094301259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : configmap references non-existent config key: service-ca.crt Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103818 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 20:59:46.103915 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.103845 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.103835639 +0000 UTC m=+39.094382139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : secret "router-metrics-certs-default" not found Jul 23 20:59:46.177892 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.177791 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ffvc4" event={"ID":"e724dd31-c672-47de-b929-30bb27fe25ef","Type":"ContainerStarted","Data":"a7f30a05abe400f1eae259df875cc774859e721dea34d0389d01a0788c3a27c5"} Jul 23 20:59:46.184068 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.183982 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" event={"ID":"43b49123-f63a-45cf-a5e9-12aa4f0eff24","Type":"ContainerStarted","Data":"32a8272320d1002597b9d2795548d37d0e5f411f074736bf8ca908cfbc4edce6"} Jul 23 20:59:46.187748 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.187659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" event={"ID":"2565a914-16d9-4bf9-8adb-f07b5bcaaff4","Type":"ContainerStarted","Data":"720a9feaa4634db1ab8870a3d41cecabec480b7f4596944a1f63c99456a6f9cb"} Jul 23 20:59:46.190959 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.190926 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" event={"ID":"3114df95-e113-40a9-a92e-dbf21f91ff1b","Type":"ContainerStarted","Data":"32cec1f02a9834ef2f2ed2f46687257e0d114f17c79850adffbb6ee58a2afc22"} Jul 23 20:59:46.197038 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.197008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" event={"ID":"9fd59ff0-d9ae-4220-b427-0a17ed800988","Type":"ContainerStarted","Data":"d6504fb87987d5142d1e9b1a5c11b91e377e4157c0c164a6cf828dd1f23fe78a"} Jul 23 20:59:46.205038 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.205000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:46.205248 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.205093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:46.205310 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.205271 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 20:59:46.205357 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.205339 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert podName:d06fea94-cf37-4c37-b739-350370a4699c nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.205319703 +0000 UTC m=+39.195866220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9tsh9" (UID: "d06fea94-cf37-4c37-b739-350370a4699c") : secret "networking-console-plugin-cert" not found Jul 23 20:59:46.205580 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.205413 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 20:59:46.205580 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:46.205452 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls podName:ccf68472-95b3-4bf4-acf8-25ecb1726591 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:48.205439934 +0000 UTC m=+39.195986436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls") pod "dns-default-pmf6p" (UID: "ccf68472-95b3-4bf4-acf8-25ecb1726591") : secret "dns-default-metrics-tls" not found Jul 23 20:59:46.214280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.213037 2578 generic.go:356] "Generic (PLEG): container finished" podID="549e7963-6203-406f-9c29-2716303002d5" containerID="0dc55520ba1e5a99870db6d24c62e9aef681b557a3974e047180489e873047f4" exitCode=0 Jul 23 20:59:46.214280 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.213133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerDied","Data":"0dc55520ba1e5a99870db6d24c62e9aef681b557a3974e047180489e873047f4"} Jul 23 20:59:46.223931 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.223846 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8g727" event={"ID":"66d92b23-a2bf-442c-9986-0a10dcd10d7e","Type":"ContainerStarted","Data":"de1f7c62061bf51cb0fcd023c5fc3f15733ff6b0db9be4be87e8b0f0da31e833"} Jul 23 20:59:46.226469 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:46.226443 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" event={"ID":"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c","Type":"ContainerStarted","Data":"4383fe0c56e3463214b547275fcbca92fbd209901f6bdfed1a2f37586973c88d"} Jul 23 20:59:47.239841 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:47.239539 2578 generic.go:356] "Generic (PLEG): container finished" podID="549e7963-6203-406f-9c29-2716303002d5" containerID="47092517cd2b4f32643d766c8eb7821b9204e9beaf6dfe3aef01f4f4407997ac" exitCode=0 Jul 23 20:59:47.239841 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:47.239623 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerDied","Data":"47092517cd2b4f32643d766c8eb7821b9204e9beaf6dfe3aef01f4f4407997ac"} Jul 23 20:59:48.125410 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.125218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:48.125652 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.125425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:48.125652 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.125571 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 20:59:48.125652 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.125605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:48.125652 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.125636 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.125617843 +0000 UTC m=+43.116164344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : secret "router-metrics-certs-default" not found Jul 23 20:59:48.125890 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.125716 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 20:59:48.125890 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.125728 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96c8965f-8wk9l: secret "image-registry-tls" not found Jul 23 20:59:48.125890 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.125761 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls podName:a886bb36-fb28-4d64-95a7-fbeb2873542a nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.125751433 +0000 UTC m=+43.116297948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls") pod "image-registry-96c8965f-8wk9l" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a") : secret "image-registry-tls" not found Jul 23 20:59:48.125890 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.125807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:48.125890 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.125851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:48.126140 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.125975 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:48.126140 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.126024 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls podName:c6f338b8-691d-4985-9398-0452739f7aec nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.126005365 +0000 UTC m=+43.116551870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tbs7t" (UID: "c6f338b8-691d-4985-9398-0452739f7aec") : secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:48.126140 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.126041 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 20:59:48.126140 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.126079 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls podName:8f41b4bf-7378-43f8-9f55-a63019753a5e nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.126067637 +0000 UTC m=+43.116614139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-srch4" (UID: "8f41b4bf-7378-43f8-9f55-a63019753a5e") : secret "samples-operator-tls" not found Jul 23 20:59:48.126382 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.126147 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.12613657 +0000 UTC m=+43.116683074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : configmap references non-existent config key: service-ca.crt Jul 23 20:59:48.227201 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.227158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:48.227389 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.227276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:48.228128 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.227540 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 20:59:48.228128 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.227639 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert podName:d06fea94-cf37-4c37-b739-350370a4699c nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.227615746 +0000 UTC m=+43.218162247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9tsh9" (UID: "d06fea94-cf37-4c37-b739-350370a4699c") : secret "networking-console-plugin-cert" not found Jul 23 20:59:48.228128 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.228044 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 20:59:48.228128 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:48.228092 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls podName:ccf68472-95b3-4bf4-acf8-25ecb1726591 nodeName:}" failed. No retries permitted until 2026-07-23 20:59:52.228076871 +0000 UTC m=+43.218623378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls") pod "dns-default-pmf6p" (UID: "ccf68472-95b3-4bf4-acf8-25ecb1726591") : secret "dns-default-metrics-tls" not found Jul 23 20:59:48.253043 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.252997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" event={"ID":"3114df95-e113-40a9-a92e-dbf21f91ff1b","Type":"ContainerStarted","Data":"472cbee40812c83ef8609367e918b2093eb26aab870bcfbb48b9c5ee9168593d"} Jul 23 20:59:48.258680 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.258644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" event={"ID":"549e7963-6203-406f-9c29-2716303002d5","Type":"ContainerStarted","Data":"9a9352bdf65944f868eaa6f7acf490529b78ca1069ea8d0b9b1364ef75232874"} Jul 23 20:59:48.268055 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.267988 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sx5mn" podStartSLOduration=34.476048831 podStartE2EDuration="36.267969619s" totalImagesPullingTime="1.791920788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:48.267054067 +0000 UTC m=+39.257600626" watchObservedRunningTime="2026-07-23 20:59:48.267969619 +0000 UTC m=+39.258516141" Jul 23 20:59:48.289436 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:48.289358 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n2cfh" podStartSLOduration=8.40756491 podStartE2EDuration="38.28933672s" totalImagesPullingTime="29.88177181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:48.288011352 +0000 UTC m=+39.278557897" watchObservedRunningTime="2026-07-23 20:59:48.28933672 +0000 UTC m=+39.279883243" Jul 23 20:59:52.168220 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.168175 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.168229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.168275 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.168326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168351 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168421 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.168356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168432 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96c8965f-8wk9l: secret "image-registry-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168418 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168483 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168436 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls podName:8f41b4bf-7378-43f8-9f55-a63019753a5e nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.168412146 +0000 UTC m=+51.158958667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-srch4" (UID: "8f41b4bf-7378-43f8-9f55-a63019753a5e") : secret "samples-operator-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168525 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.168507177 +0000 UTC m=+51.159053683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : configmap references non-existent config key: service-ca.crt Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168580 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls podName:a886bb36-fb28-4d64-95a7-fbeb2873542a nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.168534513 +0000 UTC m=+51.159081019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls") pod "image-registry-96c8965f-8wk9l" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a") : secret "image-registry-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168603 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls podName:c6f338b8-691d-4985-9398-0452739f7aec nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.1685939 +0000 UTC m=+51.159140406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tbs7t" (UID: "c6f338b8-691d-4985-9398-0452739f7aec") : secret "cluster-monitoring-operator-tls" not found Jul 23 20:59:52.168763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.168625 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.168616815 +0000 UTC m=+51.159163320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : secret "router-metrics-certs-default" not found Jul 23 20:59:52.269364 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.269330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 20:59:52.269583 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:52.269398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 20:59:52.269583 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.269469 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 20:59:52.269583 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.269523 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 20:59:52.269583 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.269532 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls podName:ccf68472-95b3-4bf4-acf8-25ecb1726591 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.269510881 +0000 UTC m=+51.260057384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls") pod "dns-default-pmf6p" (UID: "ccf68472-95b3-4bf4-acf8-25ecb1726591") : secret "dns-default-metrics-tls" not found Jul 23 20:59:52.269824 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:52.269614 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert podName:d06fea94-cf37-4c37-b739-350370a4699c nodeName:}" failed. No retries permitted until 2026-07-23 21:00:00.269595083 +0000 UTC m=+51.260141597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9tsh9" (UID: "d06fea94-cf37-4c37-b739-350370a4699c") : secret "networking-console-plugin-cert" not found Jul 23 20:59:53.277985 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:53.277941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8g727" event={"ID":"66d92b23-a2bf-442c-9986-0a10dcd10d7e","Type":"ContainerStarted","Data":"086ef95525965dfcd67ce26614ae87f34719a605c0d116767d537941a52855a3"} Jul 23 20:59:53.280427 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:53.280388 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ffvc4" event={"ID":"e724dd31-c672-47de-b929-30bb27fe25ef","Type":"ContainerStarted","Data":"19210081d5e2ab5d9704625d3f391b0a3f262694da42a5cb21b50b3fcd5eabb1"} Jul 23 20:59:53.283483 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:53.283397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" event={"ID":"43b49123-f63a-45cf-a5e9-12aa4f0eff24","Type":"ContainerStarted","Data":"5896c644ee2ec71011a6eceef476fbc3b0d7dd3725ad584b203297262a07c5e2"} Jul 23 20:59:53.285201 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:53.285174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" event={"ID":"2565a914-16d9-4bf9-8adb-f07b5bcaaff4","Type":"ContainerStarted","Data":"64bdb133bda1d09a48d90d426108a30ce4e63e30efb51a5a6dd94afa99a49e13"} Jul 23 20:59:53.294773 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:53.294204 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-8g727" podStartSLOduration=33.794924023 podStartE2EDuration="41.294182079s" totalImagesPullingTime="7.499258056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:53.29355621 +0000 UTC m=+44.284102725" watchObservedRunningTime="2026-07-23 20:59:53.294182079 +0000 UTC m=+44.284728602" Jul 23 20:59:53.309825 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:53.309740 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ffvc4" podStartSLOduration=1.801629856 podStartE2EDuration="9.309699288s" totalImagesPullingTime="7.508069432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:53.308448538 +0000 UTC m=+44.298995061" watchObservedRunningTime="2026-07-23 20:59:53.309699288 +0000 UTC m=+44.300245812" Jul 23 20:59:54.296035 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.296002 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/0.log" Jul 23 20:59:54.296507 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.296046 2578 generic.go:356] "Generic (PLEG): container finished" podID="43b49123-f63a-45cf-a5e9-12aa4f0eff24" containerID="5896c644ee2ec71011a6eceef476fbc3b0d7dd3725ad584b203297262a07c5e2" exitCode=255 Jul 23 20:59:54.296507 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.296131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" event={"ID":"43b49123-f63a-45cf-a5e9-12aa4f0eff24","Type":"ContainerDied","Data":"5896c644ee2ec71011a6eceef476fbc3b0d7dd3725ad584b203297262a07c5e2"} Jul 23 20:59:54.296507 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.296429 2578 scope.go:117] "RemoveContainer" containerID="5896c644ee2ec71011a6eceef476fbc3b0d7dd3725ad584b203297262a07c5e2" Jul 23 20:59:54.298811 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.298750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" event={"ID":"9fd59ff0-d9ae-4220-b427-0a17ed800988","Type":"ContainerStarted","Data":"004981a84801e4a6f6e4ba443c947c111a26a3c766a61e0a7be210fccbe6434b"} Jul 23 20:59:54.300246 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.300219 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" event={"ID":"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c","Type":"ContainerStarted","Data":"976ed6628d8e63982bde637320b7dbcf83e03c14798c87fb650c66e9cfbd4814"} Jul 23 20:59:54.331917 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.331861 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" podStartSLOduration=33.808389236 podStartE2EDuration="41.331837844s" totalImagesPullingTime="7.523448608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:54.331192952 +0000 UTC m=+45.321739473" watchObservedRunningTime="2026-07-23 20:59:54.331837844 +0000 UTC m=+45.322384347" Jul 23 20:59:54.368327 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.368268 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zm8xj" podStartSLOduration=33.84261473 podStartE2EDuration="41.368244275s" totalImagesPullingTime="7.525629545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:54.350447606 +0000 UTC m=+45.340994127" watchObservedRunningTime="2026-07-23 20:59:54.368244275 +0000 UTC m=+45.358790800" Jul 23 20:59:54.368881 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.368846 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" podStartSLOduration=33.83147675 podStartE2EDuration="41.368834863s" totalImagesPullingTime="7.537358113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:59:54.367838669 +0000 UTC m=+45.358385193" watchObservedRunningTime="2026-07-23 20:59:54.368834863 +0000 UTC m=+45.359381389" Jul 23 20:59:54.573365 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.573307 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-74pmk"] Jul 23 20:59:54.587328 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.587296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.587975 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.587943 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-74pmk"] Jul 23 20:59:54.589629 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.589599 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 20:59:54.589748 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.589730 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fl7rl\"" Jul 23 20:59:54.590157 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.590137 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 20:59:54.691131 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.691040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-data-volume\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.691283 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.691135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nmg\" (UniqueName: \"kubernetes.io/projected/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-api-access-54nmg\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.691283 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.691182 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-crio-socket\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.691283 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.691206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.691283 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.691243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.708013 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.707971 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:54.708129 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.708048 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 20:59:54.791776 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.791730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-data-volume\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.791971 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.791926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nmg\" (UniqueName: \"kubernetes.io/projected/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-api-access-54nmg\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.792041 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.792011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-crio-socket\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.792095 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.792048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.792143 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.792107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.792143 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.792133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-data-volume\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.792247 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:54.792207 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 20:59:54.792310 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:54.792268 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-rbac-proxy-cm podName:cc6fd4de-2bbf-4c58-9af2-854e64e95d2f nodeName:}" failed. No retries permitted until 2026-07-23 20:59:55.292249118 +0000 UTC m=+46.282795622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-74pmk" (UID: "cc6fd4de-2bbf-4c58-9af2-854e64e95d2f") : configmap "kube-rbac-proxy" not found Jul 23 20:59:54.792380 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.792312 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-crio-socket\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:54.792431 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:54.792377 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 20:59:54.792487 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:54.792432 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls podName:cc6fd4de-2bbf-4c58-9af2-854e64e95d2f nodeName:}" failed. No retries permitted until 2026-07-23 20:59:55.292412467 +0000 UTC m=+46.282958976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74pmk" (UID: "cc6fd4de-2bbf-4c58-9af2-854e64e95d2f") : secret "insights-runtime-extractor-tls" not found Jul 23 20:59:54.806566 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:54.806505 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nmg\" (UniqueName: \"kubernetes.io/projected/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-api-access-54nmg\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:55.297502 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.297456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:55.297936 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.297527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:55.297936 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:55.297643 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 20:59:55.297936 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:55.297716 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls podName:cc6fd4de-2bbf-4c58-9af2-854e64e95d2f nodeName:}" failed. No retries permitted until 2026-07-23 20:59:56.297699422 +0000 UTC m=+47.288245923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74pmk" (UID: "cc6fd4de-2bbf-4c58-9af2-854e64e95d2f") : secret "insights-runtime-extractor-tls" not found Jul 23 20:59:55.298140 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.298121 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:55.304725 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.304703 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 20:59:55.305084 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.305069 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/0.log" Jul 23 20:59:55.305157 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.305100 2578 generic.go:356] "Generic (PLEG): container finished" podID="43b49123-f63a-45cf-a5e9-12aa4f0eff24" containerID="85c6fcb9801d63742dc395ff72af756db9e7cc514feffc80f022539b916c03b2" exitCode=255 Jul 23 20:59:55.305220 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.305197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" event={"ID":"43b49123-f63a-45cf-a5e9-12aa4f0eff24","Type":"ContainerDied","Data":"85c6fcb9801d63742dc395ff72af756db9e7cc514feffc80f022539b916c03b2"} Jul 23 20:59:55.305269 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.305253 2578 scope.go:117] "RemoveContainer" containerID="5896c644ee2ec71011a6eceef476fbc3b0d7dd3725ad584b203297262a07c5e2" Jul 23 20:59:55.305632 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:55.305611 2578 scope.go:117] "RemoveContainer" containerID="85c6fcb9801d63742dc395ff72af756db9e7cc514feffc80f022539b916c03b2" Jul 23 20:59:55.305784 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:55.305767 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rwvmk_openshift-console-operator(43b49123-f63a-45cf-a5e9-12aa4f0eff24)\"" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" podUID="43b49123-f63a-45cf-a5e9-12aa4f0eff24" Jul 23 20:59:56.174282 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:56.174247 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-72tl8_2b959712-8ea5-4d5c-97c8-c53de4c55ad9/dns-node-resolver/0.log" Jul 23 20:59:56.309325 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:56.309251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:56.309763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:56.309399 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 20:59:56.309763 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:56.309459 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls podName:cc6fd4de-2bbf-4c58-9af2-854e64e95d2f nodeName:}" failed. No retries permitted until 2026-07-23 20:59:58.309440996 +0000 UTC m=+49.299987497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74pmk" (UID: "cc6fd4de-2bbf-4c58-9af2-854e64e95d2f") : secret "insights-runtime-extractor-tls" not found Jul 23 20:59:56.310503 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:56.310481 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 20:59:56.310838 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:56.310824 2578 scope.go:117] "RemoveContainer" containerID="85c6fcb9801d63742dc395ff72af756db9e7cc514feffc80f022539b916c03b2" Jul 23 20:59:56.310982 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:56.310965 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rwvmk_openshift-console-operator(43b49123-f63a-45cf-a5e9-12aa4f0eff24)\"" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" podUID="43b49123-f63a-45cf-a5e9-12aa4f0eff24" Jul 23 20:59:57.174233 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:57.174204 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d8j56_b22b0230-000b-4fd7-8596-b5832724cecb/node-ca/0.log" Jul 23 20:59:57.775059 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:57.775031 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ffvc4_e724dd31-c672-47de-b929-30bb27fe25ef/serve-healthcheck-canary/0.log" Jul 23 20:59:58.327712 ip-10-0-129-43 kubenswrapper[2578]: I0723 20:59:58.327664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 20:59:58.327874 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:58.327813 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 20:59:58.327945 ip-10-0-129-43 kubenswrapper[2578]: E0723 20:59:58.327878 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls podName:cc6fd4de-2bbf-4c58-9af2-854e64e95d2f nodeName:}" failed. No retries permitted until 2026-07-23 21:00:02.327863574 +0000 UTC m=+53.318410078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74pmk" (UID: "cc6fd4de-2bbf-4c58-9af2-854e64e95d2f") : secret "insights-runtime-extractor-tls" not found Jul 23 21:00:00.246479 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.246432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246605 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246626 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96c8965f-8wk9l: secret "image-registry-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.246602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246677 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls podName:a886bb36-fb28-4d64-95a7-fbeb2873542a nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.246659398 +0000 UTC m=+67.237205902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls") pod "image-registry-96c8965f-8wk9l" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a") : secret "image-registry-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246696 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.246717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246749 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls podName:8f41b4bf-7378-43f8-9f55-a63019753a5e nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.24673282 +0000 UTC m=+67.237279335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-srch4" (UID: "8f41b4bf-7378-43f8-9f55-a63019753a5e") : secret "samples-operator-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.246769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246787 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246828 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls podName:c6f338b8-691d-4985-9398-0452739f7aec nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.246816424 +0000 UTC m=+67.237362924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tbs7t" (UID: "c6f338b8-691d-4985-9398-0452739f7aec") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.246846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246884 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.246874611 +0000 UTC m=+67.237421126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : configmap references non-existent config key: service-ca.crt Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246936 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:00:00.246979 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.246968 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs podName:e76bfe9e-26d6-4966-8069-cc0ffddcdef9 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.246958182 +0000 UTC m=+67.237504690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs") pod "router-default-6d54bbc4cb-htvqd" (UID: "e76bfe9e-26d6-4966-8069-cc0ffddcdef9") : secret "router-metrics-certs-default" not found Jul 23 21:00:00.347267 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.347225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 21:00:00.347467 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.347369 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 21:00:00.347467 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:00.347389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 21:00:00.347467 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.347431 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert podName:d06fea94-cf37-4c37-b739-350370a4699c nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.34741556 +0000 UTC m=+67.337962060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9tsh9" (UID: "d06fea94-cf37-4c37-b739-350370a4699c") : secret "networking-console-plugin-cert" not found Jul 23 21:00:00.347643 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.347487 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:00:00.347643 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:00.347562 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls podName:ccf68472-95b3-4bf4-acf8-25ecb1726591 nodeName:}" failed. No retries permitted until 2026-07-23 21:00:16.347530053 +0000 UTC m=+67.338076568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls") pod "dns-default-pmf6p" (UID: "ccf68472-95b3-4bf4-acf8-25ecb1726591") : secret "dns-default-metrics-tls" not found Jul 23 21:00:02.366104 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:02.366063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 21:00:02.366504 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:02.366228 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:00:02.366504 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:00:02.366318 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls podName:cc6fd4de-2bbf-4c58-9af2-854e64e95d2f nodeName:}" failed. No retries permitted until 2026-07-23 21:00:10.366301675 +0000 UTC m=+61.356848178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74pmk" (UID: "cc6fd4de-2bbf-4c58-9af2-854e64e95d2f") : secret "insights-runtime-extractor-tls" not found Jul 23 21:00:04.708165 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:04.708123 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 21:00:04.708724 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:04.708196 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 21:00:04.708724 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:04.708595 2578 scope.go:117] "RemoveContainer" containerID="85c6fcb9801d63742dc395ff72af756db9e7cc514feffc80f022539b916c03b2" Jul 23 21:00:05.349001 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:05.348976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 21:00:05.349165 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:05.349031 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" event={"ID":"43b49123-f63a-45cf-a5e9-12aa4f0eff24","Type":"ContainerStarted","Data":"b1b8cdaa6e92f799befe70986bfcf6d3834b9a2eccdae50deae1c92b1f552915"} Jul 23 21:00:05.349320 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:05.349301 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 21:00:05.353964 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:05.353937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" Jul 23 21:00:05.369604 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:05.369535 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-rwvmk" podStartSLOduration=45.857794162 podStartE2EDuration="53.36951713s" totalImagesPullingTime="7.511722968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:05.368529651 +0000 UTC m=+56.359076174" watchObservedRunningTime="2026-07-23 21:00:05.36951713 +0000 UTC m=+56.360063653" Jul 23 21:00:09.166731 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:09.166700 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mr549" Jul 23 21:00:10.441735 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:10.441689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 21:00:10.443957 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:10.443935 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc6fd4de-2bbf-4c58-9af2-854e64e95d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74pmk\" (UID: \"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f\") " pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 21:00:10.503663 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:10.503633 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fl7rl\"" Jul 23 21:00:10.511626 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:10.511593 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-74pmk" Jul 23 21:00:10.639782 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:10.639631 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-74pmk"] Jul 23 21:00:10.642734 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:10.642702 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6fd4de_2bbf_4c58_9af2_854e64e95d2f.slice/crio-24bc8e9dc0b3fa1f63d635d3ca9b24839c83100b72043445e24c1613538cd342 WatchSource:0}: Error finding container 24bc8e9dc0b3fa1f63d635d3ca9b24839c83100b72043445e24c1613538cd342: Status 404 returned error can't find the container with id 24bc8e9dc0b3fa1f63d635d3ca9b24839c83100b72043445e24c1613538cd342 Jul 23 21:00:11.371143 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:11.371060 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74pmk" event={"ID":"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f","Type":"ContainerStarted","Data":"24bc8e9dc0b3fa1f63d635d3ca9b24839c83100b72043445e24c1613538cd342"} Jul 23 21:00:15.691809 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.691761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 21:00:15.694462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.694441 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 21:00:15.704491 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.704459 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3229463-d919-4499-9c87-61ab3de151ff-metrics-certs\") pod \"network-metrics-daemon-6r9p8\" (UID: \"f3229463-d919-4499-9c87-61ab3de151ff\") " pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 21:00:15.793111 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.793068 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 21:00:15.795582 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.795539 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 21:00:15.805600 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.805569 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6a4f6f0-c1d5-462f-ab87-eefaa2943c56-original-pull-secret\") pod \"global-pull-secret-syncer-bs7mm\" (UID: \"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56\") " pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 21:00:15.893703 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.893662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 21:00:15.896070 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.896040 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7gf5\" (UniqueName: \"kubernetes.io/projected/da30a5a0-f5ab-4000-9c29-c7c829d7b7ab-kube-api-access-r7gf5\") pod \"network-check-target-vcsvx\" (UID: \"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab\") " pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 21:00:15.917133 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.917095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bs7mm" Jul 23 21:00:15.925695 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.925666 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-btkpz\"" Jul 23 21:00:15.934040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:15.934013 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r9p8" Jul 23 21:00:16.020881 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.020852 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9qkcv\"" Jul 23 21:00:16.029622 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.029591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 21:00:16.212180 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.212128 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6r9p8"] Jul 23 21:00:16.215488 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:16.215457 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3229463_d919_4499_9c87_61ab3de151ff.slice/crio-42202bdbd9810c947a4c2021d2b9b3fa4040b5ae9b76e24027cbac839e55eca2 WatchSource:0}: Error finding container 42202bdbd9810c947a4c2021d2b9b3fa4040b5ae9b76e24027cbac839e55eca2: Status 404 returned error can't find the container with id 42202bdbd9810c947a4c2021d2b9b3fa4040b5ae9b76e24027cbac839e55eca2 Jul 23 21:00:16.298380 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.298337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 21:00:16.298380 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.298379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 21:00:16.298672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.298399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:16.298672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.298430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:16.298672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.298450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:00:16.299226 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.299198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-service-ca-bundle\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:16.300961 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.300936 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f41b4bf-7378-43f8-9f55-a63019753a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-srch4\" (UID: \"8f41b4bf-7378-43f8-9f55-a63019753a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 21:00:16.300961 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.300951 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6f338b8-691d-4985-9398-0452739f7aec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tbs7t\" (UID: \"c6f338b8-691d-4985-9398-0452739f7aec\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 21:00:16.301236 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.301219 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e76bfe9e-26d6-4966-8069-cc0ffddcdef9-metrics-certs\") pod \"router-default-6d54bbc4cb-htvqd\" (UID: \"e76bfe9e-26d6-4966-8069-cc0ffddcdef9\") " pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:16.301236 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.301229 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"image-registry-96c8965f-8wk9l\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:00:16.388847 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.388815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74pmk" event={"ID":"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f","Type":"ContainerStarted","Data":"d68cb9b3d051e922104fb5e97b50d48ec996148e0f76ac5805c2e44bac686b48"} Jul 23 21:00:16.389886 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.389857 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6r9p8" event={"ID":"f3229463-d919-4499-9c87-61ab3de151ff","Type":"ContainerStarted","Data":"42202bdbd9810c947a4c2021d2b9b3fa4040b5ae9b76e24027cbac839e55eca2"} Jul 23 21:00:16.399320 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.399289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 21:00:16.399414 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.399348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 21:00:16.401795 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.401767 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d06fea94-cf37-4c37-b739-350370a4699c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9tsh9\" (UID: \"d06fea94-cf37-4c37-b739-350370a4699c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 21:00:16.401795 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.401786 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ccf68472-95b3-4bf4-acf8-25ecb1726591-metrics-tls\") pod \"dns-default-pmf6p\" (UID: \"ccf68472-95b3-4bf4-acf8-25ecb1726591\") " pod="openshift-dns/dns-default-pmf6p" Jul 23 21:00:16.434138 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.434101 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vcsvx"] Jul 23 21:00:16.437499 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:16.437426 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda30a5a0_f5ab_4000_9c29_c7c829d7b7ab.slice/crio-602fd8e4fe9edfaac4cba2c6585e10252c796e541f0bb196a40f49f77676629d WatchSource:0}: Error finding container 602fd8e4fe9edfaac4cba2c6585e10252c796e541f0bb196a40f49f77676629d: Status 404 returned error can't find the container with id 602fd8e4fe9edfaac4cba2c6585e10252c796e541f0bb196a40f49f77676629d Jul 23 21:00:16.438007 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.437988 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bs7mm"] Jul 23 21:00:16.440830 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:16.440803 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a4f6f0_c1d5_462f_ab87_eefaa2943c56.slice/crio-bc4aa6d1b883d07a2d419ce2b90bbd3b8dd7e086d15b770ea4b49b7cce34048c WatchSource:0}: Error finding container bc4aa6d1b883d07a2d419ce2b90bbd3b8dd7e086d15b770ea4b49b7cce34048c: Status 404 returned error can't find the container with id bc4aa6d1b883d07a2d419ce2b90bbd3b8dd7e086d15b770ea4b49b7cce34048c Jul 23 21:00:16.447510 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.447493 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gc9km\"" Jul 23 21:00:16.456555 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.456521 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" Jul 23 21:00:16.476886 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.476863 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7xbv6\"" Jul 23 21:00:16.485944 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.485918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:16.492604 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.492572 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-km8vb\"" Jul 23 21:00:16.500655 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.500627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:00:16.516101 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.516072 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zkktg\"" Jul 23 21:00:16.523923 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.523773 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" Jul 23 21:00:16.523923 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.523877 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bv2bf\"" Jul 23 21:00:16.531948 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.531890 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" Jul 23 21:00:16.579810 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.579772 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dj8rg\"" Jul 23 21:00:16.588395 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.588048 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmf6p" Jul 23 21:00:16.644226 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.642795 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc"] Jul 23 21:00:16.649044 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.648913 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.652642 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.652579 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t"] Jul 23 21:00:16.653771 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.653286 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 21:00:16.653771 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.653609 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 23 21:00:16.655802 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.655174 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 21:00:16.655802 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.655630 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 21:00:16.682839 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.663828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc"] Jul 23 21:00:16.749667 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.747159 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-2lvbw"] Jul 23 21:00:16.752436 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.751179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:16.757818 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.755232 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 21:00:16.770611 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.765472 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 21:00:16.770611 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.768315 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-2lvbw"] Jul 23 21:00:16.773803 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.773635 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-w9ddd\"" Jul 23 21:00:16.805345 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.805107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5959a4f1-c90a-454f-a75d-f2994c691d53-klusterlet-config\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.805345 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.805189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw7gf\" (UniqueName: \"kubernetes.io/projected/5959a4f1-c90a-454f-a75d-f2994c691d53-kube-api-access-gw7gf\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.805345 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.805231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5959a4f1-c90a-454f-a75d-f2994c691d53-tmp\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.817830 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.813709 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d54bbc4cb-htvqd"] Jul 23 21:00:16.822594 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.822568 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9"] Jul 23 21:00:16.831268 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:16.830621 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06fea94_cf37_4c37_b739_350370a4699c.slice/crio-d1eccac72e9f68f83f22c4e3b733e49bca14e20f31eccd455569733f3953f8f9 WatchSource:0}: Error finding container d1eccac72e9f68f83f22c4e3b733e49bca14e20f31eccd455569733f3953f8f9: Status 404 returned error can't find the container with id d1eccac72e9f68f83f22c4e3b733e49bca14e20f31eccd455569733f3953f8f9 Jul 23 21:00:16.847728 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.847631 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4"] Jul 23 21:00:16.885814 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.885784 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96c8965f-8wk9l"] Jul 23 21:00:16.886024 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.885853 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmf6p"] Jul 23 21:00:16.908409 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:16.904900 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda886bb36_fb28_4d64_95a7_fbeb2873542a.slice/crio-d3231babab80b04d0c0fdf309dd93e67542dcdbae87f3b4d4a3899b2c9aba449 WatchSource:0}: Error finding container d3231babab80b04d0c0fdf309dd93e67542dcdbae87f3b4d4a3899b2c9aba449: Status 404 returned error can't find the container with id d3231babab80b04d0c0fdf309dd93e67542dcdbae87f3b4d4a3899b2c9aba449 Jul 23 21:00:16.908409 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.906221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6527\" (UniqueName: \"kubernetes.io/projected/b7d53cd9-bf05-4224-baa3-012ba7c85d61-kube-api-access-p6527\") pod \"downloads-5bc67dcd46-2lvbw\" (UID: \"b7d53cd9-bf05-4224-baa3-012ba7c85d61\") " pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:16.908409 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.906271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw7gf\" (UniqueName: \"kubernetes.io/projected/5959a4f1-c90a-454f-a75d-f2994c691d53-kube-api-access-gw7gf\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.908409 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.906318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5959a4f1-c90a-454f-a75d-f2994c691d53-tmp\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.908409 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.906377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5959a4f1-c90a-454f-a75d-f2994c691d53-klusterlet-config\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.908409 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.906943 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5959a4f1-c90a-454f-a75d-f2994c691d53-tmp\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.913186 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.913131 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5959a4f1-c90a-454f-a75d-f2994c691d53-klusterlet-config\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.941594 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.941529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw7gf\" (UniqueName: \"kubernetes.io/projected/5959a4f1-c90a-454f-a75d-f2994c691d53-kube-api-access-gw7gf\") pod \"klusterlet-addon-workmgr-d856949df-z7fvc\" (UID: \"5959a4f1-c90a-454f-a75d-f2994c691d53\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:16.996244 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:16.996055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:17.007890 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.007850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6527\" (UniqueName: \"kubernetes.io/projected/b7d53cd9-bf05-4224-baa3-012ba7c85d61-kube-api-access-p6527\") pod \"downloads-5bc67dcd46-2lvbw\" (UID: \"b7d53cd9-bf05-4224-baa3-012ba7c85d61\") " pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:17.021584 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.021489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6527\" (UniqueName: \"kubernetes.io/projected/b7d53cd9-bf05-4224-baa3-012ba7c85d61-kube-api-access-p6527\") pod \"downloads-5bc67dcd46-2lvbw\" (UID: \"b7d53cd9-bf05-4224-baa3-012ba7c85d61\") " pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:17.074716 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.074390 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:17.232300 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.232102 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc"] Jul 23 21:00:17.248111 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:17.248023 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5959a4f1_c90a_454f_a75d_f2994c691d53.slice/crio-6e7063ce6a614847db2e45b163411bdc669cbe94ca6da76d56e23475d106af6f WatchSource:0}: Error finding container 6e7063ce6a614847db2e45b163411bdc669cbe94ca6da76d56e23475d106af6f: Status 404 returned error can't find the container with id 6e7063ce6a614847db2e45b163411bdc669cbe94ca6da76d56e23475d106af6f Jul 23 21:00:17.301614 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.301492 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-2lvbw"] Jul 23 21:00:17.308357 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:17.308302 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d53cd9_bf05_4224_baa3_012ba7c85d61.slice/crio-5d11748e3b1a78c3d8d82736224669e05d0b65d2e60089ba61166bd4de8b6bf3 WatchSource:0}: Error finding container 5d11748e3b1a78c3d8d82736224669e05d0b65d2e60089ba61166bd4de8b6bf3: Status 404 returned error can't find the container with id 5d11748e3b1a78c3d8d82736224669e05d0b65d2e60089ba61166bd4de8b6bf3 Jul 23 21:00:17.414378 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.414339 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" event={"ID":"8f41b4bf-7378-43f8-9f55-a63019753a5e","Type":"ContainerStarted","Data":"49535a5255a8cd4309d3db2edc955cdf7b91eda79495f7c778e5493b1690b90a"} Jul 23 21:00:17.418563 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.416986 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmf6p" event={"ID":"ccf68472-95b3-4bf4-acf8-25ecb1726591","Type":"ContainerStarted","Data":"9bf5481566a7def47af3346299a9201f3f5bb9747031596c9269b17e69926f98"} Jul 23 21:00:17.431582 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.428997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" event={"ID":"d06fea94-cf37-4c37-b739-350370a4699c","Type":"ContainerStarted","Data":"d1eccac72e9f68f83f22c4e3b733e49bca14e20f31eccd455569733f3953f8f9"} Jul 23 21:00:17.434845 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.434212 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" event={"ID":"e76bfe9e-26d6-4966-8069-cc0ffddcdef9","Type":"ContainerStarted","Data":"2184a610cc7060e30c2333d0185372ab7e9153cbc9af716b71a972f18a394aee"} Jul 23 21:00:17.434845 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.434282 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" event={"ID":"e76bfe9e-26d6-4966-8069-cc0ffddcdef9","Type":"ContainerStarted","Data":"9b8924a9de44da3e41fdb952b46eafcbfa14dd432af95a15c1adb6f1c1b544d8"} Jul 23 21:00:17.443692 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.443658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vcsvx" event={"ID":"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab","Type":"ContainerStarted","Data":"3bf1fa612a273b7946c352849e00eb09dd875a0049901dc1caa3055a142704aa"} Jul 23 21:00:17.443833 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.443710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vcsvx" event={"ID":"da30a5a0-f5ab-4000-9c29-c7c829d7b7ab","Type":"ContainerStarted","Data":"602fd8e4fe9edfaac4cba2c6585e10252c796e541f0bb196a40f49f77676629d"} Jul 23 21:00:17.444237 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.444216 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 21:00:17.446870 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.446838 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" event={"ID":"5959a4f1-c90a-454f-a75d-f2994c691d53","Type":"ContainerStarted","Data":"6e7063ce6a614847db2e45b163411bdc669cbe94ca6da76d56e23475d106af6f"} Jul 23 21:00:17.452468 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.452370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" event={"ID":"c6f338b8-691d-4985-9398-0452739f7aec","Type":"ContainerStarted","Data":"6a0bcba6929610129562c0c7dabe6639f99d5321d6a4bee66ca3d2f67858432c"} Jul 23 21:00:17.453843 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.453792 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" podStartSLOduration=65.453774069 podStartE2EDuration="1m5.453774069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:17.453311184 +0000 UTC m=+68.443857736" watchObservedRunningTime="2026-07-23 21:00:17.453774069 +0000 UTC m=+68.444320593" Jul 23 21:00:17.465283 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.465242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bs7mm" event={"ID":"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56","Type":"ContainerStarted","Data":"bc4aa6d1b883d07a2d419ce2b90bbd3b8dd7e086d15b770ea4b49b7cce34048c"} Jul 23 21:00:17.481776 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.479474 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vcsvx" podStartSLOduration=67.479449383 podStartE2EDuration="1m7.479449383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:17.478172685 +0000 UTC m=+68.468719209" watchObservedRunningTime="2026-07-23 21:00:17.479449383 +0000 UTC m=+68.469995907" Jul 23 21:00:17.489010 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.482838 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-2lvbw" event={"ID":"b7d53cd9-bf05-4224-baa3-012ba7c85d61","Type":"ContainerStarted","Data":"5d11748e3b1a78c3d8d82736224669e05d0b65d2e60089ba61166bd4de8b6bf3"} Jul 23 21:00:17.489010 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.487610 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:17.490258 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.489516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" event={"ID":"a886bb36-fb28-4d64-95a7-fbeb2873542a","Type":"ContainerStarted","Data":"df4ac1e21cf2b5c81756cbf41e166c3f7c9356ac3100de431f7eb2c76a042338"} Jul 23 21:00:17.490258 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.489610 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" event={"ID":"a886bb36-fb28-4d64-95a7-fbeb2873542a","Type":"ContainerStarted","Data":"d3231babab80b04d0c0fdf309dd93e67542dcdbae87f3b4d4a3899b2c9aba449"} Jul 23 21:00:17.490258 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.489782 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:00:17.490993 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.490938 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:17.529666 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:17.528430 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" podStartSLOduration=67.528407091 podStartE2EDuration="1m7.528407091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:17.508968216 +0000 UTC m=+68.499514782" watchObservedRunningTime="2026-07-23 21:00:17.528407091 +0000 UTC m=+68.518953616" Jul 23 21:00:18.500513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:18.500445 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:18.502941 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:18.502784 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d54bbc4cb-htvqd" Jul 23 21:00:29.556325 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.556284 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" event={"ID":"8f41b4bf-7378-43f8-9f55-a63019753a5e","Type":"ContainerStarted","Data":"9e7f5fca7f99ce7bd2cf3cdb4a611e431da6cde78f9be5171188c3757808734d"} Jul 23 21:00:29.556821 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.556399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" event={"ID":"8f41b4bf-7378-43f8-9f55-a63019753a5e","Type":"ContainerStarted","Data":"5ad57a76945346573577302e90c96e428c2680a94b0d4e5f4e86e5e190234ca0"} Jul 23 21:00:29.558965 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.558912 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmf6p" event={"ID":"ccf68472-95b3-4bf4-acf8-25ecb1726591","Type":"ContainerStarted","Data":"c7198948076478bdfd7e988324a55fa96b08182428c6c1aafc931c27321badc2"} Jul 23 21:00:29.559128 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.558979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pmf6p" Jul 23 21:00:29.559128 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.559005 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmf6p" event={"ID":"ccf68472-95b3-4bf4-acf8-25ecb1726591","Type":"ContainerStarted","Data":"48291f4a83173f0f3b15d066e3895b558844b3e87df851bbb6de3c43c413341a"} Jul 23 21:00:29.560616 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.560585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" event={"ID":"d06fea94-cf37-4c37-b739-350370a4699c","Type":"ContainerStarted","Data":"8f3532500a93513e3edaf17d342cbc51be8bfc51f31e92ae68d695d74540ba65"} Jul 23 21:00:29.563199 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.562803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" event={"ID":"5959a4f1-c90a-454f-a75d-f2994c691d53","Type":"ContainerStarted","Data":"58a52cfda56e19014898eb82b35228e85511190c726285d0c214f2f1f7b0b9a7"} Jul 23 21:00:29.563199 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.562988 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:29.565296 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.564913 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" event={"ID":"c6f338b8-691d-4985-9398-0452739f7aec","Type":"ContainerStarted","Data":"c1ad3c072b8a21550bdd057eb4e723aa505d9d10bd08bb3bbf00affe93615304"} Jul 23 21:00:29.565296 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.565147 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" Jul 23 21:00:29.568104 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.568072 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bs7mm" event={"ID":"a6a4f6f0-c1d5-462f-ab87-eefaa2943c56","Type":"ContainerStarted","Data":"8c05665cc1b579c233ef53242eafa1726683240510df1357576fccf463a51ade"} Jul 23 21:00:29.571435 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.571362 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74pmk" event={"ID":"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f","Type":"ContainerStarted","Data":"6bfe5a9d10854ff45073b301dd878bfe38e2caa4e11f504d436e79f7a77722fd"} Jul 23 21:00:29.576719 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.575691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6r9p8" event={"ID":"f3229463-d919-4499-9c87-61ab3de151ff","Type":"ContainerStarted","Data":"1fabf692cbaab057d904bee9234e7e3ec2a89b9908a1de10cb6e2ed4a5d03b69"} Jul 23 21:00:29.578541 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.576989 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-srch4" podStartSLOduration=66.092828321 podStartE2EDuration="1m17.576968697s" totalImagesPullingTime="11.484140376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.575816187 +0000 UTC m=+80.566362720" watchObservedRunningTime="2026-07-23 21:00:29.576968697 +0000 UTC m=+80.567515221" Jul 23 21:00:29.578732 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.578701 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6r9p8" event={"ID":"f3229463-d919-4499-9c87-61ab3de151ff","Type":"ContainerStarted","Data":"c059eb42dd794d4825825f4960177bdd113607f30118ff3b49c11eb5aeb663c7"} Jul 23 21:00:29.598536 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.598447 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tbs7t" podStartSLOduration=67.400796549 podStartE2EDuration="1m17.59842349s" totalImagesPullingTime="10.197626941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.597154152 +0000 UTC m=+80.587700704" watchObservedRunningTime="2026-07-23 21:00:29.59842349 +0000 UTC m=+80.588970013" Jul 23 21:00:29.615438 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.615373 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9tsh9" podStartSLOduration=38.564662984 podStartE2EDuration="48.615350519s" totalImagesPullingTime="10.050687535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.614329108 +0000 UTC m=+80.604875648" watchObservedRunningTime="2026-07-23 21:00:29.615350519 +0000 UTC m=+80.605897069" Jul 23 21:00:29.633357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.632965 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pmf6p" podStartSLOduration=34.120830446 podStartE2EDuration="45.632943153s" totalImagesPullingTime="11.512112707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.63276259 +0000 UTC m=+80.623309134" watchObservedRunningTime="2026-07-23 21:00:29.632943153 +0000 UTC m=+80.623489678" Jul 23 21:00:29.652191 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.652004 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bs7mm" podStartSLOduration=67.64450541 podStartE2EDuration="1m19.651986859s" totalImagesPullingTime="12.007481449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.650680472 +0000 UTC m=+80.641227007" watchObservedRunningTime="2026-07-23 21:00:29.651986859 +0000 UTC m=+80.642533382" Jul 23 21:00:29.672284 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:29.672093 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d856949df-z7fvc" podStartSLOduration=2.44382675 podStartE2EDuration="13.672070846s" totalImagesPullingTime="11.228244096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.669893262 +0000 UTC m=+80.660439787" watchObservedRunningTime="2026-07-23 21:00:29.672070846 +0000 UTC m=+80.662617372" Jul 23 21:00:36.506498 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:36.506337 2578 patch_prober.go:28] interesting pod/image-registry-96c8965f-8wk9l container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 21:00:36.506498 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:36.506407 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" podUID="a886bb36-fb28-4d64-95a7-fbeb2873542a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 21:00:38.505169 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:38.505128 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:00:38.528287 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:38.528228 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6r9p8" podStartSLOduration=81.449789058 podStartE2EDuration="1m28.528213111s" totalImagesPullingTime="7.078424053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:29.688868575 +0000 UTC m=+80.679415099" watchObservedRunningTime="2026-07-23 21:00:38.528213111 +0000 UTC m=+89.518759633" Jul 23 21:00:39.582884 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.582847 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pmf6p" Jul 23 21:00:39.624445 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.624409 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74pmk" event={"ID":"cc6fd4de-2bbf-4c58-9af2-854e64e95d2f","Type":"ContainerStarted","Data":"e9686cd313450a6825cc65926c64be1de719e793dfbf9043227eb1ddfe4f5d31"} Jul 23 21:00:39.626224 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.626146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-2lvbw" event={"ID":"b7d53cd9-bf05-4224-baa3-012ba7c85d61","Type":"ContainerStarted","Data":"245833a8f21914249468ded337562ec4ee58cccba412279574c3fe3009c15f7e"} Jul 23 21:00:39.626367 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.626280 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:39.627852 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.627823 2578 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-2lvbw container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" start-of-body= Jul 23 21:00:39.627992 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.627878 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-2lvbw" podUID="b7d53cd9-bf05-4224-baa3-012ba7c85d61" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" Jul 23 21:00:39.643199 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.643113 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-74pmk" podStartSLOduration=17.248402434 podStartE2EDuration="45.64309184s" totalImagesPullingTime="28.394689406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:59:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:39.642903794 +0000 UTC m=+90.633450315" watchObservedRunningTime="2026-07-23 21:00:39.64309184 +0000 UTC m=+90.633638365" Jul 23 21:00:39.659351 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:39.659294 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-2lvbw" podStartSLOduration=1.5817496439999998 podStartE2EDuration="23.659277464s" totalImagesPullingTime="22.07752782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:39.65805383 +0000 UTC m=+90.648600353" watchObservedRunningTime="2026-07-23 21:00:39.659277464 +0000 UTC m=+90.649823987" Jul 23 21:00:40.530254 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:40.530212 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96c8965f-8wk9l"] Jul 23 21:00:40.645475 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:40.645440 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-2lvbw" Jul 23 21:00:41.675738 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.675701 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sjdst"] Jul 23 21:00:41.697366 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.697331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.699965 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.699936 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 21:00:41.700119 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.700006 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 21:00:41.700119 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.700030 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pfj68\"" Jul 23 21:00:41.700240 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.700162 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 21:00:41.700768 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.700747 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 21:00:41.844654 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.844612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-tls\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.844654 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.844655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-root\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.844958 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.844677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-wtmp\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.844958 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.844780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.844958 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.844892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.845110 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.844967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-metrics-client-ca\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.845110 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.845028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-sys\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.845110 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.845056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-textfile\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.845110 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.845085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9wd2\" (UniqueName: \"kubernetes.io/projected/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-kube-api-access-v9wd2\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946531 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-sys\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-textfile\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9wd2\" (UniqueName: \"kubernetes.io/projected/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-kube-api-access-v9wd2\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-tls\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946663 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-sys\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-root\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946734 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-root\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-wtmp\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946860 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-wtmp\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.946978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.946887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-metrics-client-ca\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.947524 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.947473 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.947848 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.947821 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-textfile\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.948120 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.948093 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-metrics-client-ca\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.949727 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.949703 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-tls\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.949832 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.949729 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:41.962256 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:41.962228 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9wd2\" (UniqueName: \"kubernetes.io/projected/3b2cf885-4c02-4b56-9da3-7ee4f97694e0-kube-api-access-v9wd2\") pod \"node-exporter-sjdst\" (UID: \"3b2cf885-4c02-4b56-9da3-7ee4f97694e0\") " pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:42.010344 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:42.010267 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sjdst" Jul 23 21:00:42.039436 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:00:42.039394 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2cf885_4c02_4b56_9da3_7ee4f97694e0.slice/crio-f744fc51b24f968b19c1ca01003ea0e298143dd99fbd4fc491ce31fd601fa8ee WatchSource:0}: Error finding container f744fc51b24f968b19c1ca01003ea0e298143dd99fbd4fc491ce31fd601fa8ee: Status 404 returned error can't find the container with id f744fc51b24f968b19c1ca01003ea0e298143dd99fbd4fc491ce31fd601fa8ee Jul 23 21:00:42.643252 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:42.643205 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjdst" event={"ID":"3b2cf885-4c02-4b56-9da3-7ee4f97694e0","Type":"ContainerStarted","Data":"f744fc51b24f968b19c1ca01003ea0e298143dd99fbd4fc491ce31fd601fa8ee"} Jul 23 21:00:44.654602 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:44.654539 2578 generic.go:356] "Generic (PLEG): container finished" podID="3b2cf885-4c02-4b56-9da3-7ee4f97694e0" containerID="08c0b9531bc45a573c775b8a6264bcbc8a0f963d94953278f85cfb6e1744b5bf" exitCode=0 Jul 23 21:00:44.655073 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:44.654652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjdst" event={"ID":"3b2cf885-4c02-4b56-9da3-7ee4f97694e0","Type":"ContainerDied","Data":"08c0b9531bc45a573c775b8a6264bcbc8a0f963d94953278f85cfb6e1744b5bf"} Jul 23 21:00:45.663081 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:45.663046 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjdst" event={"ID":"3b2cf885-4c02-4b56-9da3-7ee4f97694e0","Type":"ContainerStarted","Data":"ddff21c711225e423afb29641260995d98e2ca5c4a694f3f17dfa96fc8c116fa"} Jul 23 21:00:45.663672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:45.663636 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjdst" event={"ID":"3b2cf885-4c02-4b56-9da3-7ee4f97694e0","Type":"ContainerStarted","Data":"04912f72caa2b2037d40c5eac9f7852978f40c23a87f674c60d1944e42516c2e"} Jul 23 21:00:45.682204 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:45.682149 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sjdst" podStartSLOduration=2.990562035 podStartE2EDuration="4.682124934s" totalImagesPullingTime="1.588253713s" totalInitContainerRuntime="103.309186ms" isStatefulPod=true podCreationTimestamp="2026-07-23 21:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:00:45.681241264 +0000 UTC m=+96.671787816" watchObservedRunningTime="2026-07-23 21:00:45.682124934 +0000 UTC m=+96.672671458" Jul 23 21:00:49.507528 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:00:49.507492 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vcsvx" Jul 23 21:01:05.559398 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.559330 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" podUID="a886bb36-fb28-4d64-95a7-fbeb2873542a" containerName="registry" containerID="cri-o://df4ac1e21cf2b5c81756cbf41e166c3f7c9356ac3100de431f7eb2c76a042338" gracePeriod=30 Jul 23 21:01:05.743643 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.743605 2578 generic.go:356] "Generic (PLEG): container finished" podID="a886bb36-fb28-4d64-95a7-fbeb2873542a" containerID="df4ac1e21cf2b5c81756cbf41e166c3f7c9356ac3100de431f7eb2c76a042338" exitCode=0 Jul 23 21:01:05.743826 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.743678 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" event={"ID":"a886bb36-fb28-4d64-95a7-fbeb2873542a","Type":"ContainerDied","Data":"df4ac1e21cf2b5c81756cbf41e166c3f7c9356ac3100de431f7eb2c76a042338"} Jul 23 21:01:05.841474 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.841395 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:01:05.957469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957427 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-image-registry-private-configuration\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957471 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-certificates\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957498 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbnnk\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-kube-api-access-mbnnk\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957526 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957600 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a886bb36-fb28-4d64-95a7-fbeb2873542a-ca-trust-extracted\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957635 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-bound-sa-token\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957679 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-trusted-ca\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.957741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.957711 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-installation-pull-secrets\") pod \"a886bb36-fb28-4d64-95a7-fbeb2873542a\" (UID: \"a886bb36-fb28-4d64-95a7-fbeb2873542a\") " Jul 23 21:01:05.958061 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.958032 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:01:05.958139 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.958110 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:01:05.960327 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.960203 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:01:05.960327 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.960264 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-kube-api-access-mbnnk" (OuterVolumeSpecName: "kube-api-access-mbnnk") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "kube-api-access-mbnnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:01:05.960327 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.960291 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:01:05.960583 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.960331 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:01:05.960583 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.960520 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:01:05.966633 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:05.966538 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a886bb36-fb28-4d64-95a7-fbeb2873542a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a886bb36-fb28-4d64-95a7-fbeb2873542a" (UID: "a886bb36-fb28-4d64-95a7-fbeb2873542a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:01:06.059123 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059084 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-image-registry-private-configuration\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059123 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059114 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-certificates\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059123 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059125 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbnnk\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-kube-api-access-mbnnk\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059123 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059134 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-registry-tls\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059381 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059142 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a886bb36-fb28-4d64-95a7-fbeb2873542a-ca-trust-extracted\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059381 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059150 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a886bb36-fb28-4d64-95a7-fbeb2873542a-bound-sa-token\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059381 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059159 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a886bb36-fb28-4d64-95a7-fbeb2873542a-trusted-ca\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.059381 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.059167 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a886bb36-fb28-4d64-95a7-fbeb2873542a-installation-pull-secrets\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:01:06.748619 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.748588 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" Jul 23 21:01:06.748619 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.748601 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96c8965f-8wk9l" event={"ID":"a886bb36-fb28-4d64-95a7-fbeb2873542a","Type":"ContainerDied","Data":"d3231babab80b04d0c0fdf309dd93e67542dcdbae87f3b4d4a3899b2c9aba449"} Jul 23 21:01:06.749219 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.748657 2578 scope.go:117] "RemoveContainer" containerID="df4ac1e21cf2b5c81756cbf41e166c3f7c9356ac3100de431f7eb2c76a042338" Jul 23 21:01:06.766855 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.766822 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96c8965f-8wk9l"] Jul 23 21:01:06.771416 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:06.771384 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-96c8965f-8wk9l"] Jul 23 21:01:08.007747 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:08.007704 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a886bb36-fb28-4d64-95a7-fbeb2873542a" path="/var/lib/kubelet/pods/a886bb36-fb28-4d64-95a7-fbeb2873542a/volumes" Jul 23 21:01:09.762026 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:09.761991 2578 generic.go:356] "Generic (PLEG): container finished" podID="66d92b23-a2bf-442c-9986-0a10dcd10d7e" containerID="086ef95525965dfcd67ce26614ae87f34719a605c0d116767d537941a52855a3" exitCode=0 Jul 23 21:01:09.762026 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:09.762033 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8g727" event={"ID":"66d92b23-a2bf-442c-9986-0a10dcd10d7e","Type":"ContainerDied","Data":"086ef95525965dfcd67ce26614ae87f34719a605c0d116767d537941a52855a3"} Jul 23 21:01:09.762449 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:09.762424 2578 scope.go:117] "RemoveContainer" containerID="086ef95525965dfcd67ce26614ae87f34719a605c0d116767d537941a52855a3" Jul 23 21:01:10.769288 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:10.769235 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8g727" event={"ID":"66d92b23-a2bf-442c-9986-0a10dcd10d7e","Type":"ContainerStarted","Data":"21b31da6de1edee31c41f6e1ba7d757f1e2cb26d13831b7706354071ed76fc08"} Jul 23 21:01:23.824002 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:23.823969 2578 generic.go:356] "Generic (PLEG): container finished" podID="2565a914-16d9-4bf9-8adb-f07b5bcaaff4" containerID="64bdb133bda1d09a48d90d426108a30ce4e63e30efb51a5a6dd94afa99a49e13" exitCode=0 Jul 23 21:01:23.824434 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:23.824030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" event={"ID":"2565a914-16d9-4bf9-8adb-f07b5bcaaff4","Type":"ContainerDied","Data":"64bdb133bda1d09a48d90d426108a30ce4e63e30efb51a5a6dd94afa99a49e13"} Jul 23 21:01:23.824434 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:23.824329 2578 scope.go:117] "RemoveContainer" containerID="64bdb133bda1d09a48d90d426108a30ce4e63e30efb51a5a6dd94afa99a49e13" Jul 23 21:01:24.830572 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:24.830527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4b6xx" event={"ID":"2565a914-16d9-4bf9-8adb-f07b5bcaaff4","Type":"ContainerStarted","Data":"03110475e7943617c902d84805bef311979fc41181eb91b21b78cbd16ef17657"} Jul 23 21:01:29.850839 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:29.850802 2578 generic.go:356] "Generic (PLEG): container finished" podID="2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c" containerID="976ed6628d8e63982bde637320b7dbcf83e03c14798c87fb650c66e9cfbd4814" exitCode=0 Jul 23 21:01:29.851254 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:29.850878 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" event={"ID":"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c","Type":"ContainerDied","Data":"976ed6628d8e63982bde637320b7dbcf83e03c14798c87fb650c66e9cfbd4814"} Jul 23 21:01:29.851302 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:29.851253 2578 scope.go:117] "RemoveContainer" containerID="976ed6628d8e63982bde637320b7dbcf83e03c14798c87fb650c66e9cfbd4814" Jul 23 21:01:30.855948 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:01:30.855908 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-h4d5l" event={"ID":"2cd2a3fa-d46b-4bb6-bd36-b5a1a9922a3c","Type":"ContainerStarted","Data":"a0ed791f3c1c5ac159034442e8456a4e6d71bd36647b3f38c7be265286d60393"} Jul 23 21:03:08.290467 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.290433 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48"] Jul 23 21:03:08.290938 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.290724 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a886bb36-fb28-4d64-95a7-fbeb2873542a" containerName="registry" Jul 23 21:03:08.290938 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.290736 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a886bb36-fb28-4d64-95a7-fbeb2873542a" containerName="registry" Jul 23 21:03:08.290938 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.290788 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a886bb36-fb28-4d64-95a7-fbeb2873542a" containerName="registry" Jul 23 21:03:08.292770 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.292754 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.296180 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.296155 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-x85mb\"" Jul 23 21:03:08.296728 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.296712 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 21:03:08.297446 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.297430 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 21:03:08.306715 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.306683 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48"] Jul 23 21:03:08.386311 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.386277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.386311 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.386318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rms8\" (UniqueName: \"kubernetes.io/projected/731696ff-a7d0-47e4-ae53-f454eded3299-kube-api-access-5rms8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.386526 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.386350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.487479 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.487435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.487622 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.487486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rms8\" (UniqueName: \"kubernetes.io/projected/731696ff-a7d0-47e4-ae53-f454eded3299-kube-api-access-5rms8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.487622 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.487519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.487934 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.487915 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.487975 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.487936 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.497003 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.496976 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rms8\" (UniqueName: \"kubernetes.io/projected/731696ff-a7d0-47e4-ae53-f454eded3299-kube-api-access-5rms8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.602203 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.602105 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:08.734763 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:08.734726 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48"] Jul 23 21:03:08.737827 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:03:08.737793 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731696ff_a7d0_47e4_ae53_f454eded3299.slice/crio-2e7c27cb676b5516f3b08ac8d0511c7ef3c006fa58b7dfcebd128a0af6269e31 WatchSource:0}: Error finding container 2e7c27cb676b5516f3b08ac8d0511c7ef3c006fa58b7dfcebd128a0af6269e31: Status 404 returned error can't find the container with id 2e7c27cb676b5516f3b08ac8d0511c7ef3c006fa58b7dfcebd128a0af6269e31 Jul 23 21:03:09.216430 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:09.216392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" event={"ID":"731696ff-a7d0-47e4-ae53-f454eded3299","Type":"ContainerStarted","Data":"2e7c27cb676b5516f3b08ac8d0511c7ef3c006fa58b7dfcebd128a0af6269e31"} Jul 23 21:03:16.245261 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:16.245159 2578 generic.go:356] "Generic (PLEG): container finished" podID="731696ff-a7d0-47e4-ae53-f454eded3299" containerID="6f5e42b9e42a33b7db25ea9d3472e6bc96fdbb776875fea2d71396d10282518c" exitCode=0 Jul 23 21:03:16.245717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:16.245259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" event={"ID":"731696ff-a7d0-47e4-ae53-f454eded3299","Type":"ContainerDied","Data":"6f5e42b9e42a33b7db25ea9d3472e6bc96fdbb776875fea2d71396d10282518c"} Jul 23 21:03:19.258781 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:19.258742 2578 generic.go:356] "Generic (PLEG): container finished" podID="731696ff-a7d0-47e4-ae53-f454eded3299" containerID="7396a261f0f3d5fe42ab67fe47ed3cd228a1440325a8525a9477ead911c7dd36" exitCode=0 Jul 23 21:03:19.259158 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:19.258814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" event={"ID":"731696ff-a7d0-47e4-ae53-f454eded3299","Type":"ContainerDied","Data":"7396a261f0f3d5fe42ab67fe47ed3cd228a1440325a8525a9477ead911c7dd36"} Jul 23 21:03:27.296558 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:27.296512 2578 generic.go:356] "Generic (PLEG): container finished" podID="731696ff-a7d0-47e4-ae53-f454eded3299" containerID="ccbf9f4d64d5d66f980151fe758f55f27b23472d87a558c4e25bf165baa597ad" exitCode=0 Jul 23 21:03:27.296918 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:27.296594 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" event={"ID":"731696ff-a7d0-47e4-ae53-f454eded3299","Type":"ContainerDied","Data":"ccbf9f4d64d5d66f980151fe758f55f27b23472d87a558c4e25bf165baa597ad"} Jul 23 21:03:28.428883 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.428846 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:28.562036 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.561915 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-bundle\") pod \"731696ff-a7d0-47e4-ae53-f454eded3299\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " Jul 23 21:03:28.562036 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.561981 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rms8\" (UniqueName: \"kubernetes.io/projected/731696ff-a7d0-47e4-ae53-f454eded3299-kube-api-access-5rms8\") pod \"731696ff-a7d0-47e4-ae53-f454eded3299\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " Jul 23 21:03:28.562036 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.562021 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-util\") pod \"731696ff-a7d0-47e4-ae53-f454eded3299\" (UID: \"731696ff-a7d0-47e4-ae53-f454eded3299\") " Jul 23 21:03:28.562812 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.562771 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-bundle" (OuterVolumeSpecName: "bundle") pod "731696ff-a7d0-47e4-ae53-f454eded3299" (UID: "731696ff-a7d0-47e4-ae53-f454eded3299"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:03:28.564263 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.564230 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731696ff-a7d0-47e4-ae53-f454eded3299-kube-api-access-5rms8" (OuterVolumeSpecName: "kube-api-access-5rms8") pod "731696ff-a7d0-47e4-ae53-f454eded3299" (UID: "731696ff-a7d0-47e4-ae53-f454eded3299"). InnerVolumeSpecName "kube-api-access-5rms8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:03:28.566462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.566438 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-util" (OuterVolumeSpecName: "util") pod "731696ff-a7d0-47e4-ae53-f454eded3299" (UID: "731696ff-a7d0-47e4-ae53-f454eded3299"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:03:28.663471 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.663425 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:03:28.663471 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.663460 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rms8\" (UniqueName: \"kubernetes.io/projected/731696ff-a7d0-47e4-ae53-f454eded3299-kube-api-access-5rms8\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:03:28.663471 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:28.663473 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/731696ff-a7d0-47e4-ae53-f454eded3299-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:03:29.305445 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:29.305406 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" event={"ID":"731696ff-a7d0-47e4-ae53-f454eded3299","Type":"ContainerDied","Data":"2e7c27cb676b5516f3b08ac8d0511c7ef3c006fa58b7dfcebd128a0af6269e31"} Jul 23 21:03:29.305445 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:29.305427 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n7m48" Jul 23 21:03:29.305445 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:29.305455 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e7c27cb676b5516f3b08ac8d0511c7ef3c006fa58b7dfcebd128a0af6269e31" Jul 23 21:03:36.380670 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380588 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69"] Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380854 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="util" Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380866 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="util" Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380882 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="pull" Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380887 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="pull" Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380902 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="extract" Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380909 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="extract" Jul 23 21:03:36.381095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.380955 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="731696ff-a7d0-47e4-ae53-f454eded3299" containerName="extract" Jul 23 21:03:36.382877 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.382858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.386265 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.386242 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:03:36.386265 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.386245 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 21:03:36.386455 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.386246 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xb2gf\"" Jul 23 21:03:36.406353 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.406319 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69"] Jul 23 21:03:36.528737 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.528682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acfdd7e1-9092-4a93-b385-bb44a66309ea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5sj69\" (UID: \"acfdd7e1-9092-4a93-b385-bb44a66309ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.528932 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.528756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bnb6\" (UniqueName: \"kubernetes.io/projected/acfdd7e1-9092-4a93-b385-bb44a66309ea-kube-api-access-5bnb6\") pod \"cert-manager-operator-controller-manager-55c444bf86-5sj69\" (UID: \"acfdd7e1-9092-4a93-b385-bb44a66309ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.629614 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.629573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acfdd7e1-9092-4a93-b385-bb44a66309ea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5sj69\" (UID: \"acfdd7e1-9092-4a93-b385-bb44a66309ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.629810 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.629641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bnb6\" (UniqueName: \"kubernetes.io/projected/acfdd7e1-9092-4a93-b385-bb44a66309ea-kube-api-access-5bnb6\") pod \"cert-manager-operator-controller-manager-55c444bf86-5sj69\" (UID: \"acfdd7e1-9092-4a93-b385-bb44a66309ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.629981 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.629960 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acfdd7e1-9092-4a93-b385-bb44a66309ea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5sj69\" (UID: \"acfdd7e1-9092-4a93-b385-bb44a66309ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.641875 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.641804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bnb6\" (UniqueName: \"kubernetes.io/projected/acfdd7e1-9092-4a93-b385-bb44a66309ea-kube-api-access-5bnb6\") pod \"cert-manager-operator-controller-manager-55c444bf86-5sj69\" (UID: \"acfdd7e1-9092-4a93-b385-bb44a66309ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.691908 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.691870 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" Jul 23 21:03:36.832836 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:36.832797 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69"] Jul 23 21:03:36.836226 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:03:36.836191 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacfdd7e1_9092_4a93_b385_bb44a66309ea.slice/crio-36d3e800b866dd40a7a5d45615dbf3a027e5e2eb9a24f1806c57d9ea545d831b WatchSource:0}: Error finding container 36d3e800b866dd40a7a5d45615dbf3a027e5e2eb9a24f1806c57d9ea545d831b: Status 404 returned error can't find the container with id 36d3e800b866dd40a7a5d45615dbf3a027e5e2eb9a24f1806c57d9ea545d831b Jul 23 21:03:37.335401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:37.335366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" event={"ID":"acfdd7e1-9092-4a93-b385-bb44a66309ea","Type":"ContainerStarted","Data":"36d3e800b866dd40a7a5d45615dbf3a027e5e2eb9a24f1806c57d9ea545d831b"} Jul 23 21:03:43.366021 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:43.365983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" event={"ID":"acfdd7e1-9092-4a93-b385-bb44a66309ea","Type":"ContainerStarted","Data":"e1aa6256012a5b9786e2023c5e7b7d26dd56aa8ec02c9e736f7d4c5d23baf28d"} Jul 23 21:03:43.387177 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:43.387118 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5sj69" podStartSLOduration=1.380693733 podStartE2EDuration="7.387101256s" totalImagesPullingTime="6.006407523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:03:43.385047783 +0000 UTC m=+274.375594306" watchObservedRunningTime="2026-07-23 21:03:43.387101256 +0000 UTC m=+274.377647780" Jul 23 21:03:44.111429 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.108611 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xp2ks"] Jul 23 21:03:44.112205 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.112177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.118261 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.118235 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ljhh7\"" Jul 23 21:03:44.118431 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.118411 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 21:03:44.119325 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.119305 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 21:03:44.128746 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.128713 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xp2ks"] Jul 23 21:03:44.195143 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.195103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc2hv\" (UniqueName: \"kubernetes.io/projected/643877f7-3d11-4e63-8ba6-5bb45483c7c7-kube-api-access-zc2hv\") pod \"cert-manager-webhook-6486ffd997-xp2ks\" (UID: \"643877f7-3d11-4e63-8ba6-5bb45483c7c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.195319 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.195175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/643877f7-3d11-4e63-8ba6-5bb45483c7c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xp2ks\" (UID: \"643877f7-3d11-4e63-8ba6-5bb45483c7c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.296125 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.296081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc2hv\" (UniqueName: \"kubernetes.io/projected/643877f7-3d11-4e63-8ba6-5bb45483c7c7-kube-api-access-zc2hv\") pod \"cert-manager-webhook-6486ffd997-xp2ks\" (UID: \"643877f7-3d11-4e63-8ba6-5bb45483c7c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.296296 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.296139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/643877f7-3d11-4e63-8ba6-5bb45483c7c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xp2ks\" (UID: \"643877f7-3d11-4e63-8ba6-5bb45483c7c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.308567 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.308516 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/643877f7-3d11-4e63-8ba6-5bb45483c7c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xp2ks\" (UID: \"643877f7-3d11-4e63-8ba6-5bb45483c7c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.308696 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.308618 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc2hv\" (UniqueName: \"kubernetes.io/projected/643877f7-3d11-4e63-8ba6-5bb45483c7c7-kube-api-access-zc2hv\") pod \"cert-manager-webhook-6486ffd997-xp2ks\" (UID: \"643877f7-3d11-4e63-8ba6-5bb45483c7c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.440990 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.440956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:44.576705 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:44.576675 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xp2ks"] Jul 23 21:03:44.579384 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:03:44.579356 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod643877f7_3d11_4e63_8ba6_5bb45483c7c7.slice/crio-7e53fe6ba59a08c1095a86e2ebc53beb6f639ab453f6e82cc23ad51b8e3d2dd2 WatchSource:0}: Error finding container 7e53fe6ba59a08c1095a86e2ebc53beb6f639ab453f6e82cc23ad51b8e3d2dd2: Status 404 returned error can't find the container with id 7e53fe6ba59a08c1095a86e2ebc53beb6f639ab453f6e82cc23ad51b8e3d2dd2 Jul 23 21:03:45.319350 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.319312 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc"] Jul 23 21:03:45.322280 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.322263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.324912 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.324886 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 21:03:45.325180 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.325162 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 21:03:45.325447 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.325424 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-x85mb\"" Jul 23 21:03:45.332685 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.332654 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc"] Jul 23 21:03:45.375254 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.375220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" event={"ID":"643877f7-3d11-4e63-8ba6-5bb45483c7c7","Type":"ContainerStarted","Data":"7e53fe6ba59a08c1095a86e2ebc53beb6f639ab453f6e82cc23ad51b8e3d2dd2"} Jul 23 21:03:45.405779 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.405732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.405948 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.405804 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.405948 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.405899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mhln\" (UniqueName: \"kubernetes.io/projected/933fe25c-52ed-40f8-ac2b-47de34c0ea91-kube-api-access-2mhln\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.506975 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.506938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.507359 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.507007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mhln\" (UniqueName: \"kubernetes.io/projected/933fe25c-52ed-40f8-ac2b-47de34c0ea91-kube-api-access-2mhln\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.507359 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.507037 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.507424 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.507397 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.507424 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.507407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.515788 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.515759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mhln\" (UniqueName: \"kubernetes.io/projected/933fe25c-52ed-40f8-ac2b-47de34c0ea91-kube-api-access-2mhln\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.633652 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.633535 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:45.771702 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:45.771673 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc"] Jul 23 21:03:45.774513 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:03:45.774475 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933fe25c_52ed_40f8_ac2b_47de34c0ea91.slice/crio-f3340734c5b4dcf5c6f2ea071a134c22884e43b0013ee1b673d1a64fd2e4e0f0 WatchSource:0}: Error finding container f3340734c5b4dcf5c6f2ea071a134c22884e43b0013ee1b673d1a64fd2e4e0f0: Status 404 returned error can't find the container with id f3340734c5b4dcf5c6f2ea071a134c22884e43b0013ee1b673d1a64fd2e4e0f0 Jul 23 21:03:46.380761 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:46.380727 2578 generic.go:356] "Generic (PLEG): container finished" podID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerID="be5df67f83e7715083816f4f3cd4bf73dff4bb48c7e3037e21db9ea275447a06" exitCode=0 Jul 23 21:03:46.380942 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:46.380788 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" event={"ID":"933fe25c-52ed-40f8-ac2b-47de34c0ea91","Type":"ContainerDied","Data":"be5df67f83e7715083816f4f3cd4bf73dff4bb48c7e3037e21db9ea275447a06"} Jul 23 21:03:46.380942 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:46.380825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" event={"ID":"933fe25c-52ed-40f8-ac2b-47de34c0ea91","Type":"ContainerStarted","Data":"f3340734c5b4dcf5c6f2ea071a134c22884e43b0013ee1b673d1a64fd2e4e0f0"} Jul 23 21:03:54.421627 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:54.421589 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" event={"ID":"643877f7-3d11-4e63-8ba6-5bb45483c7c7","Type":"ContainerStarted","Data":"fd1c6512a58bdee1989dd41036882822879bb2bf29459edb0dd826116b79e25d"} Jul 23 21:03:54.422066 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:54.421670 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:03:54.438211 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:54.438148 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" podStartSLOduration=1.160759889 podStartE2EDuration="10.438127993s" totalImagesPullingTime="9.277368104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:03:54.43645392 +0000 UTC m=+285.427000443" watchObservedRunningTime="2026-07-23 21:03:54.438127993 +0000 UTC m=+285.428674516" Jul 23 21:03:56.432728 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:56.432692 2578 generic.go:356] "Generic (PLEG): container finished" podID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerID="eea435a0648283117557ce496b44cdb9ca455976f7783d1b14b26c074139ae85" exitCode=0 Jul 23 21:03:56.433080 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:56.432772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" event={"ID":"933fe25c-52ed-40f8-ac2b-47de34c0ea91","Type":"ContainerDied","Data":"eea435a0648283117557ce496b44cdb9ca455976f7783d1b14b26c074139ae85"} Jul 23 21:03:57.438820 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:57.438786 2578 generic.go:356] "Generic (PLEG): container finished" podID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerID="b8845bfaa239d2a116eb735c82cb615bf95efc6920faa0ef41a1320506f00a6f" exitCode=0 Jul 23 21:03:57.439222 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:57.438862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" event={"ID":"933fe25c-52ed-40f8-ac2b-47de34c0ea91","Type":"ContainerDied","Data":"b8845bfaa239d2a116eb735c82cb615bf95efc6920faa0ef41a1320506f00a6f"} Jul 23 21:03:58.578707 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.578678 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:58.720242 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.720157 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mhln\" (UniqueName: \"kubernetes.io/projected/933fe25c-52ed-40f8-ac2b-47de34c0ea91-kube-api-access-2mhln\") pod \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " Jul 23 21:03:58.720242 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.720214 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-util\") pod \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " Jul 23 21:03:58.720242 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.720240 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-bundle\") pod \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\" (UID: \"933fe25c-52ed-40f8-ac2b-47de34c0ea91\") " Jul 23 21:03:58.720720 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.720691 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-bundle" (OuterVolumeSpecName: "bundle") pod "933fe25c-52ed-40f8-ac2b-47de34c0ea91" (UID: "933fe25c-52ed-40f8-ac2b-47de34c0ea91"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:03:58.722309 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.722283 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933fe25c-52ed-40f8-ac2b-47de34c0ea91-kube-api-access-2mhln" (OuterVolumeSpecName: "kube-api-access-2mhln") pod "933fe25c-52ed-40f8-ac2b-47de34c0ea91" (UID: "933fe25c-52ed-40f8-ac2b-47de34c0ea91"). InnerVolumeSpecName "kube-api-access-2mhln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:03:58.724740 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.724721 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-util" (OuterVolumeSpecName: "util") pod "933fe25c-52ed-40f8-ac2b-47de34c0ea91" (UID: "933fe25c-52ed-40f8-ac2b-47de34c0ea91"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:03:58.821900 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.821858 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mhln\" (UniqueName: \"kubernetes.io/projected/933fe25c-52ed-40f8-ac2b-47de34c0ea91-kube-api-access-2mhln\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:03:58.821900 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.821895 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:03:58.821900 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:58.821904 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933fe25c-52ed-40f8-ac2b-47de34c0ea91-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:03:59.448980 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:59.448945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" event={"ID":"933fe25c-52ed-40f8-ac2b-47de34c0ea91","Type":"ContainerDied","Data":"f3340734c5b4dcf5c6f2ea071a134c22884e43b0013ee1b673d1a64fd2e4e0f0"} Jul 23 21:03:59.448980 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:59.448981 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85rqc" Jul 23 21:03:59.449252 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:03:59.448993 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3340734c5b4dcf5c6f2ea071a134c22884e43b0013ee1b673d1a64fd2e4e0f0" Jul 23 21:04:00.431083 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:00.431049 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xp2ks" Jul 23 21:04:04.050706 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050669 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq"] Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050951 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="extract" Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050961 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="extract" Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050975 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="util" Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050981 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="util" Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050989 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="pull" Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.050995 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="pull" Jul 23 21:04:04.051107 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.051044 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="933fe25c-52ed-40f8-ac2b-47de34c0ea91" containerName="extract" Jul 23 21:04:04.053098 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.053082 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.055640 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.055610 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ldkr8\"" Jul 23 21:04:04.055778 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.055645 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 21:04:04.056430 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.056411 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:04:04.060475 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.060441 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqjbj\" (UniqueName: \"kubernetes.io/projected/515122d9-4572-49f3-a142-6fee1d0abc26-kube-api-access-tqjbj\") pod \"openshift-lws-operator-fd8ccff4c-cgnrq\" (UID: \"515122d9-4572-49f3-a142-6fee1d0abc26\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.060601 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.060486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515122d9-4572-49f3-a142-6fee1d0abc26-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cgnrq\" (UID: \"515122d9-4572-49f3-a142-6fee1d0abc26\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.061960 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.061937 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq"] Jul 23 21:04:04.160982 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.160937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515122d9-4572-49f3-a142-6fee1d0abc26-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cgnrq\" (UID: \"515122d9-4572-49f3-a142-6fee1d0abc26\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.161163 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.161008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjbj\" (UniqueName: \"kubernetes.io/projected/515122d9-4572-49f3-a142-6fee1d0abc26-kube-api-access-tqjbj\") pod \"openshift-lws-operator-fd8ccff4c-cgnrq\" (UID: \"515122d9-4572-49f3-a142-6fee1d0abc26\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.161304 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.161284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515122d9-4572-49f3-a142-6fee1d0abc26-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cgnrq\" (UID: \"515122d9-4572-49f3-a142-6fee1d0abc26\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.169225 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.169196 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjbj\" (UniqueName: \"kubernetes.io/projected/515122d9-4572-49f3-a142-6fee1d0abc26-kube-api-access-tqjbj\") pod \"openshift-lws-operator-fd8ccff4c-cgnrq\" (UID: \"515122d9-4572-49f3-a142-6fee1d0abc26\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.362767 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.362659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" Jul 23 21:04:04.502236 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:04.502201 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq"] Jul 23 21:04:04.505560 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:04.505526 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod515122d9_4572_49f3_a142_6fee1d0abc26.slice/crio-972094725baed5dbe242c2c2616f7f482b69da5634e8c635c00dee4677ed59f2 WatchSource:0}: Error finding container 972094725baed5dbe242c2c2616f7f482b69da5634e8c635c00dee4677ed59f2: Status 404 returned error can't find the container with id 972094725baed5dbe242c2c2616f7f482b69da5634e8c635c00dee4677ed59f2 Jul 23 21:04:05.477732 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:05.477693 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" event={"ID":"515122d9-4572-49f3-a142-6fee1d0abc26","Type":"ContainerStarted","Data":"972094725baed5dbe242c2c2616f7f482b69da5634e8c635c00dee4677ed59f2"} Jul 23 21:04:06.484343 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:06.484304 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" event={"ID":"515122d9-4572-49f3-a142-6fee1d0abc26","Type":"ContainerStarted","Data":"0d7d2acb49a65e200496ba9e4bbc975491ee7aba798633500f8797322bb7c9b4"} Jul 23 21:04:06.499887 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:06.499830 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cgnrq" podStartSLOduration=0.886931725 podStartE2EDuration="2.499811437s" totalImagesPullingTime="1.612879712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:04:06.498672049 +0000 UTC m=+297.489218573" watchObservedRunningTime="2026-07-23 21:04:06.499811437 +0000 UTC m=+297.490357959" Jul 23 21:04:09.251689 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.251654 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn"] Jul 23 21:04:09.254103 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.254086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.256301 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.256271 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 21:04:09.256301 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.256271 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 21:04:09.257084 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.257062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-x85mb\"" Jul 23 21:04:09.263185 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.263159 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn"] Jul 23 21:04:09.299184 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.299134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.299384 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.299242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.299384 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.299280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7b5l\" (UniqueName: \"kubernetes.io/projected/272a0e87-0df7-489f-8798-b84210274fbb-kube-api-access-n7b5l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.400325 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.400283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.400325 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.400332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7b5l\" (UniqueName: \"kubernetes.io/projected/272a0e87-0df7-489f-8798-b84210274fbb-kube-api-access-n7b5l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.400591 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.400358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.400722 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.400702 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.400757 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.400725 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.409076 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.409049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7b5l\" (UniqueName: \"kubernetes.io/projected/272a0e87-0df7-489f-8798-b84210274fbb-kube-api-access-n7b5l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.563830 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.563725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:09.697818 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.697742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn"] Jul 23 21:04:09.700281 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:09.700250 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272a0e87_0df7_489f_8798_b84210274fbb.slice/crio-78bac918128eaea22f7e4a794c8b35302ee47eea1f79f3cd4aaf38d8933451cb WatchSource:0}: Error finding container 78bac918128eaea22f7e4a794c8b35302ee47eea1f79f3cd4aaf38d8933451cb: Status 404 returned error can't find the container with id 78bac918128eaea22f7e4a794c8b35302ee47eea1f79f3cd4aaf38d8933451cb Jul 23 21:04:09.921339 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.921277 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 21:04:09.921339 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.921325 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 21:04:09.933096 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:09.933070 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 21:04:10.502286 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:10.502248 2578 generic.go:356] "Generic (PLEG): container finished" podID="272a0e87-0df7-489f-8798-b84210274fbb" containerID="f34882f888f1ffd516e21c8f87e7605247c26018e0861043cf0cab8710ef1250" exitCode=0 Jul 23 21:04:10.502714 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:10.502307 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" event={"ID":"272a0e87-0df7-489f-8798-b84210274fbb","Type":"ContainerDied","Data":"f34882f888f1ffd516e21c8f87e7605247c26018e0861043cf0cab8710ef1250"} Jul 23 21:04:10.502714 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:10.502358 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" event={"ID":"272a0e87-0df7-489f-8798-b84210274fbb","Type":"ContainerStarted","Data":"78bac918128eaea22f7e4a794c8b35302ee47eea1f79f3cd4aaf38d8933451cb"} Jul 23 21:04:11.508615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:11.508578 2578 generic.go:356] "Generic (PLEG): container finished" podID="272a0e87-0df7-489f-8798-b84210274fbb" containerID="7e11e75e7e1abf1b73299953df215aa0d5cacc1405ce8363116900cc928f963b" exitCode=0 Jul 23 21:04:11.509103 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:11.508639 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" event={"ID":"272a0e87-0df7-489f-8798-b84210274fbb","Type":"ContainerDied","Data":"7e11e75e7e1abf1b73299953df215aa0d5cacc1405ce8363116900cc928f963b"} Jul 23 21:04:12.515633 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:12.515594 2578 generic.go:356] "Generic (PLEG): container finished" podID="272a0e87-0df7-489f-8798-b84210274fbb" containerID="086f8f29547a56100115a574c877c9bb4beee09887f744d652b3c5cd4515fd71" exitCode=0 Jul 23 21:04:12.516014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:12.515684 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" event={"ID":"272a0e87-0df7-489f-8798-b84210274fbb","Type":"ContainerDied","Data":"086f8f29547a56100115a574c877c9bb4beee09887f744d652b3c5cd4515fd71"} Jul 23 21:04:13.654846 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.654818 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:13.730227 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.730188 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-util\") pod \"272a0e87-0df7-489f-8798-b84210274fbb\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " Jul 23 21:04:13.730401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.730259 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7b5l\" (UniqueName: \"kubernetes.io/projected/272a0e87-0df7-489f-8798-b84210274fbb-kube-api-access-n7b5l\") pod \"272a0e87-0df7-489f-8798-b84210274fbb\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " Jul 23 21:04:13.730401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.730280 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-bundle\") pod \"272a0e87-0df7-489f-8798-b84210274fbb\" (UID: \"272a0e87-0df7-489f-8798-b84210274fbb\") " Jul 23 21:04:13.731082 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.731052 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-bundle" (OuterVolumeSpecName: "bundle") pod "272a0e87-0df7-489f-8798-b84210274fbb" (UID: "272a0e87-0df7-489f-8798-b84210274fbb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:04:13.732483 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.732462 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/272a0e87-0df7-489f-8798-b84210274fbb-kube-api-access-n7b5l" (OuterVolumeSpecName: "kube-api-access-n7b5l") pod "272a0e87-0df7-489f-8798-b84210274fbb" (UID: "272a0e87-0df7-489f-8798-b84210274fbb"). InnerVolumeSpecName "kube-api-access-n7b5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:04:13.735783 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.735756 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-util" (OuterVolumeSpecName: "util") pod "272a0e87-0df7-489f-8798-b84210274fbb" (UID: "272a0e87-0df7-489f-8798-b84210274fbb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:04:13.830807 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.830726 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7b5l\" (UniqueName: \"kubernetes.io/projected/272a0e87-0df7-489f-8798-b84210274fbb-kube-api-access-n7b5l\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:04:13.830807 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.830751 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:04:13.830807 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:13.830760 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272a0e87-0df7-489f-8798-b84210274fbb-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:04:14.525888 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:14.525854 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" event={"ID":"272a0e87-0df7-489f-8798-b84210274fbb","Type":"ContainerDied","Data":"78bac918128eaea22f7e4a794c8b35302ee47eea1f79f3cd4aaf38d8933451cb"} Jul 23 21:04:14.525888 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:14.525891 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsjngn" Jul 23 21:04:14.526094 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:14.525901 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78bac918128eaea22f7e4a794c8b35302ee47eea1f79f3cd4aaf38d8933451cb" Jul 23 21:04:26.907121 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907087 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr"] Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907375 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="util" Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907385 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="util" Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907398 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="pull" Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907405 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="pull" Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907412 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="extract" Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907418 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="extract" Jul 23 21:04:26.907615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.907518 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="272a0e87-0df7-489f-8798-b84210274fbb" containerName="extract" Jul 23 21:04:26.913047 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.913020 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:26.915469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.915441 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 21:04:26.915612 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.915541 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 21:04:26.915678 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.915655 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pvrrr\"" Jul 23 21:04:26.915842 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.915829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 21:04:26.915917 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.915893 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 21:04:26.924602 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.924539 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr"] Jul 23 21:04:26.939748 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.939715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d6cbff37-d565-4cef-8524-2633993b8467-manifests\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:26.939907 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.939753 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjp8l\" (UniqueName: \"kubernetes.io/projected/d6cbff37-d565-4cef-8524-2633993b8467-kube-api-access-pjp8l\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:26.939907 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.939871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d6cbff37-d565-4cef-8524-2633993b8467-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:26.939977 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.939903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6cbff37-d565-4cef-8524-2633993b8467-webhook-cert\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:26.939977 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:26.939964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6cbff37-d565-4cef-8524-2633993b8467-tmp\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.041356 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.041315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d6cbff37-d565-4cef-8524-2633993b8467-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.041356 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.041358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6cbff37-d565-4cef-8524-2633993b8467-webhook-cert\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.041637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.041421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6cbff37-d565-4cef-8524-2633993b8467-tmp\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.041637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.041462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d6cbff37-d565-4cef-8524-2633993b8467-manifests\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.041637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.041490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjp8l\" (UniqueName: \"kubernetes.io/projected/d6cbff37-d565-4cef-8524-2633993b8467-kube-api-access-pjp8l\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.041931 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.041897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d6cbff37-d565-4cef-8524-2633993b8467-manifests\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.043862 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.043831 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6cbff37-d565-4cef-8524-2633993b8467-tmp\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.043982 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.043949 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d6cbff37-d565-4cef-8524-2633993b8467-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.044196 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.044176 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6cbff37-d565-4cef-8524-2633993b8467-webhook-cert\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.053843 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.053818 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjp8l\" (UniqueName: \"kubernetes.io/projected/d6cbff37-d565-4cef-8524-2633993b8467-kube-api-access-pjp8l\") pod \"opendatahub-operator-controller-manager-77944bd489-kl9vr\" (UID: \"d6cbff37-d565-4cef-8524-2633993b8467\") " pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.224043 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.224001 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:27.376155 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.376112 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr"] Jul 23 21:04:27.379581 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:27.379537 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6cbff37_d565_4cef_8524_2633993b8467.slice/crio-556cddf65716e14624a6199bef63e5c93b1d7c27d5b63821298752a8e29d8f91 WatchSource:0}: Error finding container 556cddf65716e14624a6199bef63e5c93b1d7c27d5b63821298752a8e29d8f91: Status 404 returned error can't find the container with id 556cddf65716e14624a6199bef63e5c93b1d7c27d5b63821298752a8e29d8f91 Jul 23 21:04:27.381351 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.381334 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 21:04:27.586484 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:27.586401 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" event={"ID":"d6cbff37-d565-4cef-8524-2633993b8467","Type":"ContainerStarted","Data":"556cddf65716e14624a6199bef63e5c93b1d7c27d5b63821298752a8e29d8f91"} Jul 23 21:04:30.607531 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:30.607491 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" event={"ID":"d6cbff37-d565-4cef-8524-2633993b8467","Type":"ContainerStarted","Data":"71d2080ed8eb1ae37d7273e426d94725fab26510aa9db350f7a5a16c2f43cdc2"} Jul 23 21:04:32.618196 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:32.618148 2578 generic.go:356] "Generic (PLEG): container finished" podID="d6cbff37-d565-4cef-8524-2633993b8467" containerID="71d2080ed8eb1ae37d7273e426d94725fab26510aa9db350f7a5a16c2f43cdc2" exitCode=0 Jul 23 21:04:32.618699 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:32.618226 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" event={"ID":"d6cbff37-d565-4cef-8524-2633993b8467","Type":"ContainerDied","Data":"71d2080ed8eb1ae37d7273e426d94725fab26510aa9db350f7a5a16c2f43cdc2"} Jul 23 21:04:33.625198 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:33.625156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" event={"ID":"d6cbff37-d565-4cef-8524-2633993b8467","Type":"ContainerStarted","Data":"c3b6bb3990f180e5a8dc0c9bef310eccc5f98b2e3de2b9c2517ec3156360e903"} Jul 23 21:04:33.625657 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:33.625394 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:33.647490 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:33.647436 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" podStartSLOduration=3.075729095 podStartE2EDuration="7.647419995s" totalImagesPullingTime="3.415119764s" totalInitContainerRuntime="1.156571136s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:04:33.645424886 +0000 UTC m=+324.635971411" watchObservedRunningTime="2026-07-23 21:04:33.647419995 +0000 UTC m=+324.637966519" Jul 23 21:04:44.631979 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:44.631940 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77944bd489-kl9vr" Jul 23 21:04:46.879840 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:46.879802 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf"] Jul 23 21:04:46.889094 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:46.889064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:46.891886 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:46.891867 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 21:04:46.892013 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:46.891870 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 21:04:46.896982 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:46.896935 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf"] Jul 23 21:04:47.018925 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.018887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.019101 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.018935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9bcl\" (UniqueName: \"kubernetes.io/projected/77a9aa17-a753-4ae1-bbc9-511ad743d45d-kube-api-access-c9bcl\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.019101 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.019029 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tmp\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.086671 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.086606 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pck5h"] Jul 23 21:04:47.090223 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.090205 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.092624 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.092590 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"odh-model-controller-7998c67dc4-pck5h\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="8d78ac26-890f-49e1-92b9-9131d2bcc660" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.092789 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.092768 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"odh-model-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" type="*v1.Secret" Jul 23 21:04:47.093608 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.093588 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"odh-model-controller-dockercfg-cdqkp\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cdqkp\"" type="*v1.Secret" Jul 23 21:04:47.112318 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.112282 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pck5h"] Jul 23 21:04:47.119662 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.119628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9bcl\" (UniqueName: \"kubernetes.io/projected/77a9aa17-a753-4ae1-bbc9-511ad743d45d-kube-api-access-c9bcl\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.119820 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.119675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tmp\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.119820 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.119733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.119901 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.119832 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 23 21:04:47.119901 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.119893 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs podName:77a9aa17-a753-4ae1-bbc9-511ad743d45d nodeName:}" failed. No retries permitted until 2026-07-23 21:04:47.619871951 +0000 UTC m=+338.610418456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs") pod "kube-auth-proxy-5cc7f4f888-kg4mf" (UID: "77a9aa17-a753-4ae1-bbc9-511ad743d45d") : secret "kube-auth-proxy-tls" not found Jul 23 21:04:47.122089 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.122057 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tmp\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.135567 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.135483 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9bcl\" (UniqueName: \"kubernetes.io/projected/77a9aa17-a753-4ae1-bbc9-511ad743d45d-kube-api-access-c9bcl\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.220936 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.220892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7r5n\" (UniqueName: \"kubernetes.io/projected/8d78ac26-890f-49e1-92b9-9131d2bcc660-kube-api-access-m7r5n\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.221112 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.221020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78ac26-890f-49e1-92b9-9131d2bcc660-cert\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.322326 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.322280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7r5n\" (UniqueName: \"kubernetes.io/projected/8d78ac26-890f-49e1-92b9-9131d2bcc660-kube-api-access-m7r5n\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.322503 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.322374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78ac26-890f-49e1-92b9-9131d2bcc660-cert\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.332142 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.332109 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7r5n\" (UniqueName: \"kubernetes.io/projected/8d78ac26-890f-49e1-92b9-9131d2bcc660-kube-api-access-m7r5n\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:47.625193 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:47.625140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:47.625386 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.625306 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 23 21:04:47.625427 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:47.625386 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs podName:77a9aa17-a753-4ae1-bbc9-511ad743d45d nodeName:}" failed. No retries permitted until 2026-07-23 21:04:48.625369795 +0000 UTC m=+339.615916297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs") pod "kube-auth-proxy-5cc7f4f888-kg4mf" (UID: "77a9aa17-a753-4ae1-bbc9-511ad743d45d") : secret "kube-auth-proxy-tls" not found Jul 23 21:04:48.126180 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.126133 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-644vr"] Jul 23 21:04:48.129857 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.129830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.132483 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.132455 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 21:04:48.139608 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.139578 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-644vr"] Jul 23 21:04:48.159404 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.159364 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cdqkp\"" Jul 23 21:04:48.230741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.230695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5cf93ef-3058-484b-a2da-58e05a718cca-tls-certs\") pod \"model-serving-api-54d89bd5fc-644vr\" (UID: \"d5cf93ef-3058-484b-a2da-58e05a718cca\") " pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.230912 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.230773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9st2j\" (UniqueName: \"kubernetes.io/projected/d5cf93ef-3058-484b-a2da-58e05a718cca-kube-api-access-9st2j\") pod \"model-serving-api-54d89bd5fc-644vr\" (UID: \"d5cf93ef-3058-484b-a2da-58e05a718cca\") " pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.273532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.273496 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk"] Jul 23 21:04:48.276964 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.276945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.279449 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.279428 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 21:04:48.279719 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.279692 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2cz8n\"" Jul 23 21:04:48.279719 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.279712 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 21:04:48.279861 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.279730 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 21:04:48.279861 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.279705 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 21:04:48.288362 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.288333 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk"] Jul 23 21:04:48.323062 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:48.323018 2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: failed to sync secret cache: timed out waiting for the condition Jul 23 21:04:48.323241 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:04:48.323184 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d78ac26-890f-49e1-92b9-9131d2bcc660-cert podName:8d78ac26-890f-49e1-92b9-9131d2bcc660 nodeName:}" failed. No retries permitted until 2026-07-23 21:04:48.823095893 +0000 UTC m=+339.813642395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8d78ac26-890f-49e1-92b9-9131d2bcc660-cert") pod "odh-model-controller-7998c67dc4-pck5h" (UID: "8d78ac26-890f-49e1-92b9-9131d2bcc660") : failed to sync secret cache: timed out waiting for the condition Jul 23 21:04:48.331961 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.331929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9st2j\" (UniqueName: \"kubernetes.io/projected/d5cf93ef-3058-484b-a2da-58e05a718cca-kube-api-access-9st2j\") pod \"model-serving-api-54d89bd5fc-644vr\" (UID: \"d5cf93ef-3058-484b-a2da-58e05a718cca\") " pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.332050 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.332035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5cf93ef-3058-484b-a2da-58e05a718cca-tls-certs\") pod \"model-serving-api-54d89bd5fc-644vr\" (UID: \"d5cf93ef-3058-484b-a2da-58e05a718cca\") " pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.334620 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.334594 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5cf93ef-3058-484b-a2da-58e05a718cca-tls-certs\") pod \"model-serving-api-54d89bd5fc-644vr\" (UID: \"d5cf93ef-3058-484b-a2da-58e05a718cca\") " pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.339697 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.339672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9st2j\" (UniqueName: \"kubernetes.io/projected/d5cf93ef-3058-484b-a2da-58e05a718cca-kube-api-access-9st2j\") pod \"model-serving-api-54d89bd5fc-644vr\" (UID: \"d5cf93ef-3058-484b-a2da-58e05a718cca\") " pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.350715 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.350691 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 21:04:48.432475 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.432643 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-token\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.432643 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.432643 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.432643 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432592 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-cacerts\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.432779 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzzfp\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-kube-api-access-fzzfp\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.432779 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.432678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb1e228a-612d-4586-a0a8-1a0932d3958f-local-certs\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.442401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.442373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:48.534298 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.534462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.534462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-cacerts\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.534462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzzfp\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-kube-api-access-fzzfp\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.534700 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb1e228a-612d-4586-a0a8-1a0932d3958f-local-certs\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.534700 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.534700 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.534620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-token\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.535102 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.535044 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.537343 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.537300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb1e228a-612d-4586-a0a8-1a0932d3958f-local-certs\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.537454 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.537415 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-cacerts\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.537513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.537490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.537576 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.537524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.542540 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.542508 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-token\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.543169 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.543146 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzzfp\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-kube-api-access-fzzfp\") pod \"istiod-openshift-gateway-7574459656-9dvpk\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.578390 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.578339 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-644vr"] Jul 23 21:04:48.580654 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:48.580614 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cf93ef_3058_484b_a2da_58e05a718cca.slice/crio-7d6ba958e932447367c12426aba5bdd04bd15aa34f4fcf845faf34c25b022d45 WatchSource:0}: Error finding container 7d6ba958e932447367c12426aba5bdd04bd15aa34f4fcf845faf34c25b022d45: Status 404 returned error can't find the container with id 7d6ba958e932447367c12426aba5bdd04bd15aa34f4fcf845faf34c25b022d45 Jul 23 21:04:48.587057 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.587028 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:48.635755 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.635721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:48.638969 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.638940 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9aa17-a753-4ae1-bbc9-511ad743d45d-tls-certs\") pod \"kube-auth-proxy-5cc7f4f888-kg4mf\" (UID: \"77a9aa17-a753-4ae1-bbc9-511ad743d45d\") " pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:48.700258 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.700174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" Jul 23 21:04:48.702200 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.702146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" event={"ID":"d5cf93ef-3058-484b-a2da-58e05a718cca","Type":"ContainerStarted","Data":"7d6ba958e932447367c12426aba5bdd04bd15aa34f4fcf845faf34c25b022d45"} Jul 23 21:04:48.750441 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.750409 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk"] Jul 23 21:04:48.752393 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:48.752251 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1e228a_612d_4586_a0a8_1a0932d3958f.slice/crio-635f236fbfbbe0464eacb95c8d7718ecc08fb67a8c435526af7dfb889f962182 WatchSource:0}: Error finding container 635f236fbfbbe0464eacb95c8d7718ecc08fb67a8c435526af7dfb889f962182: Status 404 returned error can't find the container with id 635f236fbfbbe0464eacb95c8d7718ecc08fb67a8c435526af7dfb889f962182 Jul 23 21:04:48.838074 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.838037 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78ac26-890f-49e1-92b9-9131d2bcc660-cert\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:48.839819 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.839792 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf"] Jul 23 21:04:48.840541 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.840520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78ac26-890f-49e1-92b9-9131d2bcc660-cert\") pod \"odh-model-controller-7998c67dc4-pck5h\" (UID: \"8d78ac26-890f-49e1-92b9-9131d2bcc660\") " pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:48.841800 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:48.841774 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a9aa17_a753_4ae1_bbc9_511ad743d45d.slice/crio-e495e8a18d8a4faa23e97f2e4a9db169e2c3be3829c1582b4801d09fa02f94f4 WatchSource:0}: Error finding container e495e8a18d8a4faa23e97f2e4a9db169e2c3be3829c1582b4801d09fa02f94f4: Status 404 returned error can't find the container with id e495e8a18d8a4faa23e97f2e4a9db169e2c3be3829c1582b4801d09fa02f94f4 Jul 23 21:04:48.900287 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:48.900253 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:49.060203 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:49.060167 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pck5h"] Jul 23 21:04:49.063851 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:49.063824 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d78ac26_890f_49e1_92b9_9131d2bcc660.slice/crio-6c5517384b600840c243cd7299d66183510875120424f89432611372a09a507f WatchSource:0}: Error finding container 6c5517384b600840c243cd7299d66183510875120424f89432611372a09a507f: Status 404 returned error can't find the container with id 6c5517384b600840c243cd7299d66183510875120424f89432611372a09a507f Jul 23 21:04:49.714812 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:49.714731 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" event={"ID":"8d78ac26-890f-49e1-92b9-9131d2bcc660","Type":"ContainerStarted","Data":"6c5517384b600840c243cd7299d66183510875120424f89432611372a09a507f"} Jul 23 21:04:49.717218 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:49.717032 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" event={"ID":"77a9aa17-a753-4ae1-bbc9-511ad743d45d","Type":"ContainerStarted","Data":"e495e8a18d8a4faa23e97f2e4a9db169e2c3be3829c1582b4801d09fa02f94f4"} Jul 23 21:04:49.720297 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:49.720230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" event={"ID":"cb1e228a-612d-4586-a0a8-1a0932d3958f","Type":"ContainerStarted","Data":"635f236fbfbbe0464eacb95c8d7718ecc08fb67a8c435526af7dfb889f962182"} Jul 23 21:04:53.747256 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.747218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" event={"ID":"8d78ac26-890f-49e1-92b9-9131d2bcc660","Type":"ContainerStarted","Data":"f36da1206b83fa98dbcf7375c0587b2753556f154dfaddd852e8d2f689c813c6"} Jul 23 21:04:53.747727 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.747298 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:04:53.748775 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.748745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" event={"ID":"d5cf93ef-3058-484b-a2da-58e05a718cca","Type":"ContainerStarted","Data":"9c14f9e6d18c94e2ba12128a6fcef1153c68bd83591d0ade7cf6a1ae1239dd0d"} Jul 23 21:04:53.748893 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.748874 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:04:53.750381 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.750350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" event={"ID":"77a9aa17-a753-4ae1-bbc9-511ad743d45d","Type":"ContainerStarted","Data":"b7166e236809e1ae67b3cefb02e255fc972ee5f44cf7bc2079a5e1b46b33e2a3"} Jul 23 21:04:53.784924 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.784863 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" podStartSLOduration=3.177288683 podStartE2EDuration="6.784845121s" totalImagesPullingTime="3.607556438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:04:53.78279051 +0000 UTC m=+344.773337031" watchObservedRunningTime="2026-07-23 21:04:53.784845121 +0000 UTC m=+344.775391642" Jul 23 21:04:53.839209 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.839150 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" podStartSLOduration=1.763942225 podStartE2EDuration="5.839131571s" totalImagesPullingTime="4.075189346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:04:53.820914205 +0000 UTC m=+344.811460728" watchObservedRunningTime="2026-07-23 21:04:53.839131571 +0000 UTC m=+344.829678133" Jul 23 21:04:53.840272 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:53.840237 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cc7f4f888-kg4mf" podStartSLOduration=4.018350775 podStartE2EDuration="7.840226965s" totalImagesPullingTime="3.82187619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:04:53.838163749 +0000 UTC m=+344.828710308" watchObservedRunningTime="2026-07-23 21:04:53.840226965 +0000 UTC m=+344.830773521" Jul 23 21:04:57.026135 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.026091 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:04:57.026431 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.026179 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:04:57.445417 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.445380 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z"] Jul 23 21:04:57.448739 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.448718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.454827 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.454801 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-68nmr\"" Jul 23 21:04:57.455112 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.455084 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 21:04:57.469034 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.469005 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z"] Jul 23 21:04:57.503209 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.503172 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l"] Jul 23 21:04:57.506710 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.506686 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.514275 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.514252 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d884v\"" Jul 23 21:04:57.514529 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.514515 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 23 21:04:57.536612 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.536578 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l"] Jul 23 21:04:57.616832 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.616775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2bbq\" (UniqueName: \"kubernetes.io/projected/5edcce5e-7cfb-436b-b3f1-f80131b617d6-kube-api-access-n2bbq\") pod \"kserve-controller-manager-d6f4b65b8-k9z7z\" (UID: \"5edcce5e-7cfb-436b-b3f1-f80131b617d6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.617257 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.617213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/736aa336-9275-43ae-955e-5076af0b5e3f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sw25l\" (UID: \"736aa336-9275-43ae-955e-5076af0b5e3f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.617488 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.617470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqj77\" (UniqueName: \"kubernetes.io/projected/736aa336-9275-43ae-955e-5076af0b5e3f-kube-api-access-dqj77\") pod \"llmisvc-controller-manager-777c6bd7d5-sw25l\" (UID: \"736aa336-9275-43ae-955e-5076af0b5e3f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.617719 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.617679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5edcce5e-7cfb-436b-b3f1-f80131b617d6-cert\") pod \"kserve-controller-manager-d6f4b65b8-k9z7z\" (UID: \"5edcce5e-7cfb-436b-b3f1-f80131b617d6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.718998 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.718883 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5edcce5e-7cfb-436b-b3f1-f80131b617d6-cert\") pod \"kserve-controller-manager-d6f4b65b8-k9z7z\" (UID: \"5edcce5e-7cfb-436b-b3f1-f80131b617d6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.718998 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.718940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2bbq\" (UniqueName: \"kubernetes.io/projected/5edcce5e-7cfb-436b-b3f1-f80131b617d6-kube-api-access-n2bbq\") pod \"kserve-controller-manager-d6f4b65b8-k9z7z\" (UID: \"5edcce5e-7cfb-436b-b3f1-f80131b617d6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.718998 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.718995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/736aa336-9275-43ae-955e-5076af0b5e3f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sw25l\" (UID: \"736aa336-9275-43ae-955e-5076af0b5e3f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.719255 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.719015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqj77\" (UniqueName: \"kubernetes.io/projected/736aa336-9275-43ae-955e-5076af0b5e3f-kube-api-access-dqj77\") pod \"llmisvc-controller-manager-777c6bd7d5-sw25l\" (UID: \"736aa336-9275-43ae-955e-5076af0b5e3f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.721504 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.721477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/736aa336-9275-43ae-955e-5076af0b5e3f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sw25l\" (UID: \"736aa336-9275-43ae-955e-5076af0b5e3f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.721671 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.721532 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5edcce5e-7cfb-436b-b3f1-f80131b617d6-cert\") pod \"kserve-controller-manager-d6f4b65b8-k9z7z\" (UID: \"5edcce5e-7cfb-436b-b3f1-f80131b617d6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.730443 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.730409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqj77\" (UniqueName: \"kubernetes.io/projected/736aa336-9275-43ae-955e-5076af0b5e3f-kube-api-access-dqj77\") pod \"llmisvc-controller-manager-777c6bd7d5-sw25l\" (UID: \"736aa336-9275-43ae-955e-5076af0b5e3f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.730815 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.730789 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2bbq\" (UniqueName: \"kubernetes.io/projected/5edcce5e-7cfb-436b-b3f1-f80131b617d6-kube-api-access-n2bbq\") pod \"kserve-controller-manager-d6f4b65b8-k9z7z\" (UID: \"5edcce5e-7cfb-436b-b3f1-f80131b617d6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.759635 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.759507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:04:57.773701 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.773663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" event={"ID":"cb1e228a-612d-4586-a0a8-1a0932d3958f","Type":"ContainerStarted","Data":"91ed21762ad9b9864a601d5ac6e49eb85acf3b44fc38843ac616c0c8d5f9517f"} Jul 23 21:04:57.773877 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.773852 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:57.796339 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.796253 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" podStartSLOduration=1.5257243040000001 podStartE2EDuration="9.796230314s" totalImagesPullingTime="8.27050601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:04:57.795838546 +0000 UTC m=+348.786385094" watchObservedRunningTime="2026-07-23 21:04:57.796230314 +0000 UTC m=+348.786776838" Jul 23 21:04:57.816978 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.816942 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:04:57.917654 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.917582 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z"] Jul 23 21:04:57.922236 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:57.922208 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5edcce5e_7cfb_436b_b3f1_f80131b617d6.slice/crio-9df36c25a35e10f8aab63ce6893caffb82e0818145bd37ae18b71958a22923e2 WatchSource:0}: Error finding container 9df36c25a35e10f8aab63ce6893caffb82e0818145bd37ae18b71958a22923e2: Status 404 returned error can't find the container with id 9df36c25a35e10f8aab63ce6893caffb82e0818145bd37ae18b71958a22923e2 Jul 23 21:04:57.984410 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:57.984318 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l"] Jul 23 21:04:57.992468 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:57.992427 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod736aa336_9275_43ae_955e_5076af0b5e3f.slice/crio-79e6180f673ad456cbda35529eec487e9f11e33affcd611b55fe40351beb02a9 WatchSource:0}: Error finding container 79e6180f673ad456cbda35529eec487e9f11e33affcd611b55fe40351beb02a9: Status 404 returned error can't find the container with id 79e6180f673ad456cbda35529eec487e9f11e33affcd611b55fe40351beb02a9 Jul 23 21:04:58.315874 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.315796 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km"] Jul 23 21:04:58.320066 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.320044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.322754 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.322729 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xsmqs\"" Jul 23 21:04:58.332652 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.332623 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km"] Jul 23 21:04:58.425480 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425441 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0d4e90-3322-48c9-a256-2c8057a43d89-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425480 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425902 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x8l4\" (UniqueName: \"kubernetes.io/projected/0a0d4e90-3322-48c9-a256-2c8057a43d89-kube-api-access-8x8l4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425902 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.425902 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.425771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526762 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x8l4\" (UniqueName: \"kubernetes.io/projected/0a0d4e90-3322-48c9-a256-2c8057a43d89-kube-api-access-8x8l4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526762 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0d4e90-3322-48c9-a256-2c8057a43d89-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526906 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.526991 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.526940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.527345 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.527244 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.527345 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.527284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.527345 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.527327 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.527460 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.527430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.527663 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.527647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0d4e90-3322-48c9-a256-2c8057a43d89-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.529263 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.529242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.529511 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.529490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.554193 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.554156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x8l4\" (UniqueName: \"kubernetes.io/projected/0a0d4e90-3322-48c9-a256-2c8057a43d89-kube-api-access-8x8l4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.555659 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.555637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0d4e90-3322-48c9-a256-2c8057a43d89-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45j9km\" (UID: \"0a0d4e90-3322-48c9-a256-2c8057a43d89\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.632012 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.631874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:04:58.780515 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.780483 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km"] Jul 23 21:04:58.783045 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.783014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" event={"ID":"736aa336-9275-43ae-955e-5076af0b5e3f","Type":"ContainerStarted","Data":"79e6180f673ad456cbda35529eec487e9f11e33affcd611b55fe40351beb02a9"} Jul 23 21:04:58.784469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.784440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" event={"ID":"5edcce5e-7cfb-436b-b3f1-f80131b617d6","Type":"ContainerStarted","Data":"9df36c25a35e10f8aab63ce6893caffb82e0818145bd37ae18b71958a22923e2"} Jul 23 21:04:58.786777 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:58.786746 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:04:58.788446 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:04:58.788420 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0d4e90_3322_48c9_a256_2c8057a43d89.slice/crio-5baeba4926fcd11ff8a0a1d543df9fe4ab20f1e7a220bba7c3b3c727c481a9ba WatchSource:0}: Error finding container 5baeba4926fcd11ff8a0a1d543df9fe4ab20f1e7a220bba7c3b3c727c481a9ba: Status 404 returned error can't find the container with id 5baeba4926fcd11ff8a0a1d543df9fe4ab20f1e7a220bba7c3b3c727c481a9ba Jul 23 21:04:59.473337 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:59.473293 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk"] Jul 23 21:04:59.794037 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:04:59.793936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" event={"ID":"0a0d4e90-3322-48c9-a256-2c8057a43d89","Type":"ContainerStarted","Data":"5baeba4926fcd11ff8a0a1d543df9fe4ab20f1e7a220bba7c3b3c727c481a9ba"} Jul 23 21:05:00.800447 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:00.800395 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" podUID="cb1e228a-612d-4586-a0a8-1a0932d3958f" containerName="discovery" containerID="cri-o://91ed21762ad9b9864a601d5ac6e49eb85acf3b44fc38843ac616c0c8d5f9517f" gracePeriod=30 Jul 23 21:05:01.807874 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:01.807829 2578 generic.go:356] "Generic (PLEG): container finished" podID="cb1e228a-612d-4586-a0a8-1a0932d3958f" containerID="91ed21762ad9b9864a601d5ac6e49eb85acf3b44fc38843ac616c0c8d5f9517f" exitCode=0 Jul 23 21:05:01.808268 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:01.807902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" event={"ID":"cb1e228a-612d-4586-a0a8-1a0932d3958f","Type":"ContainerDied","Data":"91ed21762ad9b9864a601d5ac6e49eb85acf3b44fc38843ac616c0c8d5f9517f"} Jul 23 21:05:02.519851 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.519825 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:05:02.563812 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563657 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzzfp\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-kube-api-access-fzzfp\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.563812 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563710 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-ca-configmap\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.563812 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563737 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-cacerts\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.563812 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563770 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-kubeconfig\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.564104 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb1e228a-612d-4586-a0a8-1a0932d3958f-local-certs\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.564104 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563911 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-token\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.564104 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.563946 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-dns-cert\") pod \"cb1e228a-612d-4586-a0a8-1a0932d3958f\" (UID: \"cb1e228a-612d-4586-a0a8-1a0932d3958f\") " Jul 23 21:05:02.564244 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.564201 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:05:02.567082 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.567051 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:05:02.567193 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.567096 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:05:02.567258 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.567205 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-cacerts" (OuterVolumeSpecName: "cacerts") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:05:02.567258 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.567215 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-kube-api-access-fzzfp" (OuterVolumeSpecName: "kube-api-access-fzzfp") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "kube-api-access-fzzfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:05:02.567330 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.567285 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:05:02.567330 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.567285 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb1e228a-612d-4586-a0a8-1a0932d3958f-local-certs" (OuterVolumeSpecName: "local-certs") pod "cb1e228a-612d-4586-a0a8-1a0932d3958f" (UID: "cb1e228a-612d-4586-a0a8-1a0932d3958f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:05:02.574262 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.574229 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:05:02.574356 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.574313 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:05:02.574423 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.574358 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:05:02.665171 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665130 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-dns-cert\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.665171 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665172 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzzfp\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-kube-api-access-fzzfp\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.665357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665186 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-csr-ca-configmap\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.665357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665200 2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-cacerts\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.665357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665213 2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-kubeconfig\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.665357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665226 2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb1e228a-612d-4586-a0a8-1a0932d3958f-local-certs\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.665357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.665238 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb1e228a-612d-4586-a0a8-1a0932d3958f-istio-token\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:02.816076 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.815987 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" Jul 23 21:05:02.816076 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.815980 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" event={"ID":"cb1e228a-612d-4586-a0a8-1a0932d3958f","Type":"ContainerDied","Data":"635f236fbfbbe0464eacb95c8d7718ecc08fb67a8c435526af7dfb889f962182"} Jul 23 21:05:02.816595 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.816138 2578 scope.go:117] "RemoveContainer" containerID="91ed21762ad9b9864a601d5ac6e49eb85acf3b44fc38843ac616c0c8d5f9517f" Jul 23 21:05:02.817693 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.817657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" event={"ID":"736aa336-9275-43ae-955e-5076af0b5e3f","Type":"ContainerStarted","Data":"6d088a0372e3f22eb355dbf3bd149afc1701cf48d7b2d7b4ba0511dfdbbc3cb3"} Jul 23 21:05:02.817811 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.817713 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:05:02.819571 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.819512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" event={"ID":"5edcce5e-7cfb-436b-b3f1-f80131b617d6","Type":"ContainerStarted","Data":"bbd3b95bfcce61dcde7a6665314d614f23e568d4e42cfbf12a639a88cc209629"} Jul 23 21:05:02.819665 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.819651 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:05:02.821267 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.821247 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" event={"ID":"0a0d4e90-3322-48c9-a256-2c8057a43d89","Type":"ContainerStarted","Data":"c7b3fe0eeaa087474cf0ff03110a858b5ccb70ef3969cf49802e0ee6f4d0e967"} Jul 23 21:05:02.837455 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.837403 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" podStartSLOduration=1.246243849 podStartE2EDuration="5.837384716s" totalImagesPullingTime="4.591140867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:05:02.835483471 +0000 UTC m=+353.826029985" watchObservedRunningTime="2026-07-23 21:05:02.837384716 +0000 UTC m=+353.827931274" Jul 23 21:05:02.854071 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.854014 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" podStartSLOduration=1.32825056 podStartE2EDuration="5.853994103s" totalImagesPullingTime="4.525743543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:05:02.853637651 +0000 UTC m=+353.844184195" watchObservedRunningTime="2026-07-23 21:05:02.853994103 +0000 UTC m=+353.844540627" Jul 23 21:05:02.868799 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.868767 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk"] Jul 23 21:05:02.872792 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.872759 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk"] Jul 23 21:05:02.890970 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:02.890916 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" podStartSLOduration=1.107789953 podStartE2EDuration="4.89090071s" totalImagesPullingTime="3.783110757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:05:02.889820461 +0000 UTC m=+353.880367057" watchObservedRunningTime="2026-07-23 21:05:02.89090071 +0000 UTC m=+353.881447233" Jul 23 21:05:03.632799 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:03.632758 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:05:03.637463 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:03.637436 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:05:03.828145 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:03.828111 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:05:03.829210 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:03.829183 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45j9km" Jul 23 21:05:04.009688 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:04.009651 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb1e228a-612d-4586-a0a8-1a0932d3958f" path="/var/lib/kubelet/pods/cb1e228a-612d-4586-a0a8-1a0932d3958f/volumes" Jul 23 21:05:04.758290 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:04.758254 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pck5h" Jul 23 21:05:04.760216 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:04.760190 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-644vr" Jul 23 21:05:06.785142 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:06.785096 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-9dvpk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 21:05:06.785613 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:06.785163 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9dvpk" podUID="cb1e228a-612d-4586-a0a8-1a0932d3958f" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.32:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 21:05:33.830239 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:33.830204 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sw25l" Jul 23 21:05:33.835800 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:33.835761 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k9z7z" Jul 23 21:05:47.830643 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.830604 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wq7lq"] Jul 23 21:05:47.831089 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.830983 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1e228a-612d-4586-a0a8-1a0932d3958f" containerName="discovery" Jul 23 21:05:47.831089 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.830994 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1e228a-612d-4586-a0a8-1a0932d3958f" containerName="discovery" Jul 23 21:05:47.831089 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.831042 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1e228a-612d-4586-a0a8-1a0932d3958f" containerName="discovery" Jul 23 21:05:47.836136 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.836104 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:47.838278 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.838213 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ffsxq\"" Jul 23 21:05:47.838278 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.838236 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 21:05:47.838507 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.838356 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 21:05:47.845658 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.845623 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wq7lq"] Jul 23 21:05:47.955234 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:47.955193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnrq\" (UniqueName: \"kubernetes.io/projected/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f-kube-api-access-dgnrq\") pod \"kuadrant-operator-catalog-wq7lq\" (UID: \"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f\") " pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:48.056321 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.056272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnrq\" (UniqueName: \"kubernetes.io/projected/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f-kube-api-access-dgnrq\") pod \"kuadrant-operator-catalog-wq7lq\" (UID: \"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f\") " pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:48.064478 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.064439 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnrq\" (UniqueName: \"kubernetes.io/projected/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f-kube-api-access-dgnrq\") pod \"kuadrant-operator-catalog-wq7lq\" (UID: \"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f\") " pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:48.148449 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.148360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:48.201922 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.201839 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wq7lq"] Jul 23 21:05:48.283472 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.283444 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wq7lq"] Jul 23 21:05:48.285405 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:05:48.285375 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2d50cb_74a4_4b47_bcff_f08e10a9ec6f.slice/crio-2a8258437ef9c0d094c89d098e825ca69f1f2db846406dd6cb5e85dcb94548e4 WatchSource:0}: Error finding container 2a8258437ef9c0d094c89d098e825ca69f1f2db846406dd6cb5e85dcb94548e4: Status 404 returned error can't find the container with id 2a8258437ef9c0d094c89d098e825ca69f1f2db846406dd6cb5e85dcb94548e4 Jul 23 21:05:48.410630 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.410537 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pc5ph"] Jul 23 21:05:48.413649 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.413628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:48.422111 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.422084 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pc5ph"] Jul 23 21:05:48.460733 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.460689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kch24\" (UniqueName: \"kubernetes.io/projected/4525cdd9-63e2-45a7-b9ba-e9373d6c0b60-kube-api-access-kch24\") pod \"kuadrant-operator-catalog-pc5ph\" (UID: \"4525cdd9-63e2-45a7-b9ba-e9373d6c0b60\") " pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:48.561718 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.561677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kch24\" (UniqueName: \"kubernetes.io/projected/4525cdd9-63e2-45a7-b9ba-e9373d6c0b60-kube-api-access-kch24\") pod \"kuadrant-operator-catalog-pc5ph\" (UID: \"4525cdd9-63e2-45a7-b9ba-e9373d6c0b60\") " pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:48.569179 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.569144 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kch24\" (UniqueName: \"kubernetes.io/projected/4525cdd9-63e2-45a7-b9ba-e9373d6c0b60-kube-api-access-kch24\") pod \"kuadrant-operator-catalog-pc5ph\" (UID: \"4525cdd9-63e2-45a7-b9ba-e9373d6c0b60\") " pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:48.724441 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.724405 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:48.865146 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:48.865112 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pc5ph"] Jul 23 21:05:48.868814 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:05:48.868785 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4525cdd9_63e2_45a7_b9ba_e9373d6c0b60.slice/crio-f60c2818262a1bbfc1d9d54ffaf50c2eece1434efa241a05290ee78bab1c4730 WatchSource:0}: Error finding container f60c2818262a1bbfc1d9d54ffaf50c2eece1434efa241a05290ee78bab1c4730: Status 404 returned error can't find the container with id f60c2818262a1bbfc1d9d54ffaf50c2eece1434efa241a05290ee78bab1c4730 Jul 23 21:05:49.050094 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:49.050008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" event={"ID":"4525cdd9-63e2-45a7-b9ba-e9373d6c0b60","Type":"ContainerStarted","Data":"f60c2818262a1bbfc1d9d54ffaf50c2eece1434efa241a05290ee78bab1c4730"} Jul 23 21:05:49.051320 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:49.051290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" event={"ID":"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f","Type":"ContainerStarted","Data":"2a8258437ef9c0d094c89d098e825ca69f1f2db846406dd6cb5e85dcb94548e4"} Jul 23 21:05:51.062916 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.062870 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" event={"ID":"4525cdd9-63e2-45a7-b9ba-e9373d6c0b60","Type":"ContainerStarted","Data":"502d26170703ff097d7c4141e8b4c264c8d6685bdf88ac7f3279a7f47044e990"} Jul 23 21:05:51.064332 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.064305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" event={"ID":"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f","Type":"ContainerStarted","Data":"cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50"} Jul 23 21:05:51.064448 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.064392 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" podUID="2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" containerName="registry-server" containerID="cri-o://cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50" gracePeriod=2 Jul 23 21:05:51.079609 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.079390 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" podStartSLOduration=1.637749088 podStartE2EDuration="3.079373722s" totalImagesPullingTime="1.441624634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:05:51.077486379 +0000 UTC m=+402.068032903" watchObservedRunningTime="2026-07-23 21:05:51.079373722 +0000 UTC m=+402.069920237" Jul 23 21:05:51.093323 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.093271 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" podStartSLOduration=2.068901775 podStartE2EDuration="4.093254218s" totalImagesPullingTime="2.024352443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:05:51.09226231 +0000 UTC m=+402.082808833" watchObservedRunningTime="2026-07-23 21:05:51.093254218 +0000 UTC m=+402.083800740" Jul 23 21:05:51.321808 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.321733 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:51.386850 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.386807 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgnrq\" (UniqueName: \"kubernetes.io/projected/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f-kube-api-access-dgnrq\") pod \"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f\" (UID: \"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f\") " Jul 23 21:05:51.389129 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.389096 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f-kube-api-access-dgnrq" (OuterVolumeSpecName: "kube-api-access-dgnrq") pod "2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" (UID: "2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f"). InnerVolumeSpecName "kube-api-access-dgnrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:05:51.487526 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:51.487485 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgnrq\" (UniqueName: \"kubernetes.io/projected/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f-kube-api-access-dgnrq\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:05:52.071242 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.071208 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" containerID="cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50" exitCode=0 Jul 23 21:05:52.071751 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.071252 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" Jul 23 21:05:52.071751 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.071295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" event={"ID":"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f","Type":"ContainerDied","Data":"cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50"} Jul 23 21:05:52.071751 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.071350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wq7lq" event={"ID":"2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f","Type":"ContainerDied","Data":"2a8258437ef9c0d094c89d098e825ca69f1f2db846406dd6cb5e85dcb94548e4"} Jul 23 21:05:52.071751 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.071381 2578 scope.go:117] "RemoveContainer" containerID="cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50" Jul 23 21:05:52.080462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.080442 2578 scope.go:117] "RemoveContainer" containerID="cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50" Jul 23 21:05:52.080812 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:05:52.080781 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50\": container with ID starting with cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50 not found: ID does not exist" containerID="cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50" Jul 23 21:05:52.080913 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.080822 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50"} err="failed to get container status \"cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50\": rpc error: code = NotFound desc = could not find container \"cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50\": container with ID starting with cea5e9a193403d4e0b5ed0bd81029058bf0e921b7026e7aa3a3ccd03be956c50 not found: ID does not exist" Jul 23 21:05:52.087065 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.087034 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wq7lq"] Jul 23 21:05:52.090210 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:52.090175 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wq7lq"] Jul 23 21:05:54.009049 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:54.009014 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" path="/var/lib/kubelet/pods/2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f/volumes" Jul 23 21:05:58.724761 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:58.724716 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:58.725161 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:58.724788 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:58.748599 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:58.748566 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:05:59.127613 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:05:59.127517 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pc5ph" Jul 23 21:06:03.605682 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.605637 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x"] Jul 23 21:06:03.606069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.605997 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" containerName="registry-server" Jul 23 21:06:03.606069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.606009 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" containerName="registry-server" Jul 23 21:06:03.606148 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.606072 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c2d50cb-74a4-4b47-bcff-f08e10a9ec6f" containerName="registry-server" Jul 23 21:06:03.609428 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.609405 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.611895 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.611869 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9p2v6\"" Jul 23 21:06:03.617243 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.616836 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x"] Jul 23 21:06:03.699501 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.699459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.699501 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.699502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.699742 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.699571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56nr6\" (UniqueName: \"kubernetes.io/projected/0f0a77f6-1431-4d4a-abe8-498eb204818b-kube-api-access-56nr6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.800771 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.800724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56nr6\" (UniqueName: \"kubernetes.io/projected/0f0a77f6-1431-4d4a-abe8-498eb204818b-kube-api-access-56nr6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.800952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.800824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.800952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.800848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.801192 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.801175 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.801249 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.801229 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.809192 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.809159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56nr6\" (UniqueName: \"kubernetes.io/projected/0f0a77f6-1431-4d4a-abe8-498eb204818b-kube-api-access-56nr6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:03.920172 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:03.920065 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:04.059663 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.059636 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x"] Jul 23 21:06:04.062032 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:04.061991 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0a77f6_1431_4d4a_abe8_498eb204818b.slice/crio-5d7220a528d3d3be9724ff6b7a489dc3084568c8a8c1d272216a160cafc20fb1 WatchSource:0}: Error finding container 5d7220a528d3d3be9724ff6b7a489dc3084568c8a8c1d272216a160cafc20fb1: Status 404 returned error can't find the container with id 5d7220a528d3d3be9724ff6b7a489dc3084568c8a8c1d272216a160cafc20fb1 Jul 23 21:06:04.131499 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.131461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" event={"ID":"0f0a77f6-1431-4d4a-abe8-498eb204818b","Type":"ContainerStarted","Data":"5d7220a528d3d3be9724ff6b7a489dc3084568c8a8c1d272216a160cafc20fb1"} Jul 23 21:06:04.410585 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.410531 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4"] Jul 23 21:06:04.414012 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.413993 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.421275 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.421244 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4"] Jul 23 21:06:04.507268 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.507169 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.507268 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.507219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.507447 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.507281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9wsl\" (UniqueName: \"kubernetes.io/projected/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-kube-api-access-h9wsl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.608132 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.608087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.608132 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.608133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.608537 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.608183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9wsl\" (UniqueName: \"kubernetes.io/projected/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-kube-api-access-h9wsl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.608537 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.608485 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.608639 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.608578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.616579 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.616532 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9wsl\" (UniqueName: \"kubernetes.io/projected/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-kube-api-access-h9wsl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.724324 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.724288 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:04.865688 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:04.865658 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4"] Jul 23 21:06:04.867489 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:04.867456 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c8b1fb_2ee0_4c64_a45c_6d19bd072426.slice/crio-1f8845d6e1b80ed7b43751779b6d4e003753ba1ee2518988219f5a95e85968f2 WatchSource:0}: Error finding container 1f8845d6e1b80ed7b43751779b6d4e003753ba1ee2518988219f5a95e85968f2: Status 404 returned error can't find the container with id 1f8845d6e1b80ed7b43751779b6d4e003753ba1ee2518988219f5a95e85968f2 Jul 23 21:06:05.002452 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.002414 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk"] Jul 23 21:06:05.006350 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.006316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.013044 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.013013 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk"] Jul 23 21:06:05.112263 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.112222 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxtj\" (UniqueName: \"kubernetes.io/projected/c26bd86a-52f8-47a1-ad05-e30418215620-kube-api-access-gtxtj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.112437 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.112341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.112637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.112613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.138653 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.138614 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerID="b0be46f44689282ad8275e2f0da5d5fba04b3a07cc1fb647e8e29463f73c32be" exitCode=0 Jul 23 21:06:05.138826 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.138701 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" event={"ID":"0f0a77f6-1431-4d4a-abe8-498eb204818b","Type":"ContainerDied","Data":"b0be46f44689282ad8275e2f0da5d5fba04b3a07cc1fb647e8e29463f73c32be"} Jul 23 21:06:05.140338 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.140311 2578 generic.go:356] "Generic (PLEG): container finished" podID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerID="a0baf6a2458305193f140ba7cea6670ec089c5640c295cab316b607c7c3fea24" exitCode=0 Jul 23 21:06:05.140468 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.140400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" event={"ID":"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426","Type":"ContainerDied","Data":"a0baf6a2458305193f140ba7cea6670ec089c5640c295cab316b607c7c3fea24"} Jul 23 21:06:05.140468 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.140454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" event={"ID":"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426","Type":"ContainerStarted","Data":"1f8845d6e1b80ed7b43751779b6d4e003753ba1ee2518988219f5a95e85968f2"} Jul 23 21:06:05.213860 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.213822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.214032 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.213878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxtj\" (UniqueName: \"kubernetes.io/projected/c26bd86a-52f8-47a1-ad05-e30418215620-kube-api-access-gtxtj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.214032 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.213981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.214294 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.214272 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.214464 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.214442 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.222032 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.222006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxtj\" (UniqueName: \"kubernetes.io/projected/c26bd86a-52f8-47a1-ad05-e30418215620-kube-api-access-gtxtj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.350832 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.350737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:05.408423 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.408389 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44"] Jul 23 21:06:05.415470 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.415442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.420877 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.420843 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44"] Jul 23 21:06:05.493537 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.493507 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk"] Jul 23 21:06:05.495644 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:05.495610 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26bd86a_52f8_47a1_ad05_e30418215620.slice/crio-8e08b29b2451cb9bbec9bc8ef2f3475edd3175d4016ca4a46474610c4d5388dc WatchSource:0}: Error finding container 8e08b29b2451cb9bbec9bc8ef2f3475edd3175d4016ca4a46474610c4d5388dc: Status 404 returned error can't find the container with id 8e08b29b2451cb9bbec9bc8ef2f3475edd3175d4016ca4a46474610c4d5388dc Jul 23 21:06:05.516244 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.516209 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.516392 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.516267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtfb\" (UniqueName: \"kubernetes.io/projected/c584ce0a-50c8-43d9-8eb8-a003bc99f206-kube-api-access-gbtfb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.516392 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.516380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.617312 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.617271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.618041 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.617354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.618041 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.617390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtfb\" (UniqueName: \"kubernetes.io/projected/c584ce0a-50c8-43d9-8eb8-a003bc99f206-kube-api-access-gbtfb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.618041 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.617750 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.618041 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.617797 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.626128 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.626094 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtfb\" (UniqueName: \"kubernetes.io/projected/c584ce0a-50c8-43d9-8eb8-a003bc99f206-kube-api-access-gbtfb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.727434 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.727395 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:05.898400 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:05.898323 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44"] Jul 23 21:06:05.901691 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:05.901662 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc584ce0a_50c8_43d9_8eb8_a003bc99f206.slice/crio-167e1c11aab10335427eabc8fc18864701c1e847778aed1d5e0045c9c6ab5fc6 WatchSource:0}: Error finding container 167e1c11aab10335427eabc8fc18864701c1e847778aed1d5e0045c9c6ab5fc6: Status 404 returned error can't find the container with id 167e1c11aab10335427eabc8fc18864701c1e847778aed1d5e0045c9c6ab5fc6 Jul 23 21:06:06.147725 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.147685 2578 generic.go:356] "Generic (PLEG): container finished" podID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerID="93c2e7cb9b3c44b796110c55754c441435d3ff1f7f29d64b0741efa3a75ea450" exitCode=0 Jul 23 21:06:06.147907 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.147810 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" event={"ID":"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426","Type":"ContainerDied","Data":"93c2e7cb9b3c44b796110c55754c441435d3ff1f7f29d64b0741efa3a75ea450"} Jul 23 21:06:06.149354 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.149284 2578 generic.go:356] "Generic (PLEG): container finished" podID="c26bd86a-52f8-47a1-ad05-e30418215620" containerID="e1f5aca699f5e88c118888e016989a9e68705c9b8e201a6ce3759eb38d50f76d" exitCode=0 Jul 23 21:06:06.149469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.149356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" event={"ID":"c26bd86a-52f8-47a1-ad05-e30418215620","Type":"ContainerDied","Data":"e1f5aca699f5e88c118888e016989a9e68705c9b8e201a6ce3759eb38d50f76d"} Jul 23 21:06:06.149469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.149409 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" event={"ID":"c26bd86a-52f8-47a1-ad05-e30418215620","Type":"ContainerStarted","Data":"8e08b29b2451cb9bbec9bc8ef2f3475edd3175d4016ca4a46474610c4d5388dc"} Jul 23 21:06:06.151287 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.151261 2578 generic.go:356] "Generic (PLEG): container finished" podID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerID="c051547fcd2848ef3c021717baceaf80d36df9e03138dfda7690f6fd2f70fa59" exitCode=0 Jul 23 21:06:06.151401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.151352 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" event={"ID":"c584ce0a-50c8-43d9-8eb8-a003bc99f206","Type":"ContainerDied","Data":"c051547fcd2848ef3c021717baceaf80d36df9e03138dfda7690f6fd2f70fa59"} Jul 23 21:06:06.151401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:06.151388 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" event={"ID":"c584ce0a-50c8-43d9-8eb8-a003bc99f206","Type":"ContainerStarted","Data":"167e1c11aab10335427eabc8fc18864701c1e847778aed1d5e0045c9c6ab5fc6"} Jul 23 21:06:07.159133 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:07.159096 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerID="42d95ebca956a081ce9c49b86afea53a494de0fa6034ced036ae27d0b987dac6" exitCode=0 Jul 23 21:06:07.159528 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:07.159185 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" event={"ID":"0f0a77f6-1431-4d4a-abe8-498eb204818b","Type":"ContainerDied","Data":"42d95ebca956a081ce9c49b86afea53a494de0fa6034ced036ae27d0b987dac6"} Jul 23 21:06:07.162014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:07.161922 2578 generic.go:356] "Generic (PLEG): container finished" podID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerID="8c9a9733ed9ab639f695875d860897a128f04ac595964558ee0dfd0ee9ac1d84" exitCode=0 Jul 23 21:06:07.162014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:07.161967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" event={"ID":"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426","Type":"ContainerDied","Data":"8c9a9733ed9ab639f695875d860897a128f04ac595964558ee0dfd0ee9ac1d84"} Jul 23 21:06:08.169103 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.169062 2578 generic.go:356] "Generic (PLEG): container finished" podID="c26bd86a-52f8-47a1-ad05-e30418215620" containerID="1af75cf2a60bd6c8a3b2ca7c3aa71570df0f23bde201374411ad1bcaa2ab0de8" exitCode=0 Jul 23 21:06:08.169586 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.169150 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" event={"ID":"c26bd86a-52f8-47a1-ad05-e30418215620","Type":"ContainerDied","Data":"1af75cf2a60bd6c8a3b2ca7c3aa71570df0f23bde201374411ad1bcaa2ab0de8"} Jul 23 21:06:08.171536 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.171511 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerID="4e22b5fea1c881be1c7ca77d6fe5f3d3674fd5ad23a043e6150445d9072c6edd" exitCode=0 Jul 23 21:06:08.171647 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.171590 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" event={"ID":"0f0a77f6-1431-4d4a-abe8-498eb204818b","Type":"ContainerDied","Data":"4e22b5fea1c881be1c7ca77d6fe5f3d3674fd5ad23a043e6150445d9072c6edd"} Jul 23 21:06:08.173513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.173492 2578 generic.go:356] "Generic (PLEG): container finished" podID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerID="e3d5ea73f2d37e4fc3c84e745bcbeb430a4451725c6d2b6fa68fb26940f692f4" exitCode=0 Jul 23 21:06:08.173599 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.173578 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" event={"ID":"c584ce0a-50c8-43d9-8eb8-a003bc99f206","Type":"ContainerDied","Data":"e3d5ea73f2d37e4fc3c84e745bcbeb430a4451725c6d2b6fa68fb26940f692f4"} Jul 23 21:06:08.328371 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.328335 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:08.447150 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.447114 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9wsl\" (UniqueName: \"kubernetes.io/projected/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-kube-api-access-h9wsl\") pod \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " Jul 23 21:06:08.447325 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.447189 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-util\") pod \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " Jul 23 21:06:08.447325 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.447236 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-bundle\") pod \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\" (UID: \"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426\") " Jul 23 21:06:08.447972 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.447939 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-bundle" (OuterVolumeSpecName: "bundle") pod "d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" (UID: "d6c8b1fb-2ee0-4c64-a45c-6d19bd072426"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:08.449326 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.449301 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-kube-api-access-h9wsl" (OuterVolumeSpecName: "kube-api-access-h9wsl") pod "d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" (UID: "d6c8b1fb-2ee0-4c64-a45c-6d19bd072426"). InnerVolumeSpecName "kube-api-access-h9wsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:08.452255 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.452219 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-util" (OuterVolumeSpecName: "util") pod "d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" (UID: "d6c8b1fb-2ee0-4c64-a45c-6d19bd072426"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:08.548069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.548025 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9wsl\" (UniqueName: \"kubernetes.io/projected/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-kube-api-access-h9wsl\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:08.548069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.548059 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:08.548069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:08.548069 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6c8b1fb-2ee0-4c64-a45c-6d19bd072426-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:09.182095 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.182062 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" Jul 23 21:06:09.182567 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.182057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4" event={"ID":"d6c8b1fb-2ee0-4c64-a45c-6d19bd072426","Type":"ContainerDied","Data":"1f8845d6e1b80ed7b43751779b6d4e003753ba1ee2518988219f5a95e85968f2"} Jul 23 21:06:09.182567 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.182198 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f8845d6e1b80ed7b43751779b6d4e003753ba1ee2518988219f5a95e85968f2" Jul 23 21:06:09.184230 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.184202 2578 generic.go:356] "Generic (PLEG): container finished" podID="c26bd86a-52f8-47a1-ad05-e30418215620" containerID="cf384eacf4b6d909b644c412044e490b1316135f6e23d68551d033b73f796bb9" exitCode=0 Jul 23 21:06:09.184366 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.184292 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" event={"ID":"c26bd86a-52f8-47a1-ad05-e30418215620","Type":"ContainerDied","Data":"cf384eacf4b6d909b644c412044e490b1316135f6e23d68551d033b73f796bb9"} Jul 23 21:06:09.186375 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.186353 2578 generic.go:356] "Generic (PLEG): container finished" podID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerID="4f3734123b7aae5b98f4eec189a605a2bd071aa52824a9e0f8a226611edbb3de" exitCode=0 Jul 23 21:06:09.186506 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.186382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" event={"ID":"c584ce0a-50c8-43d9-8eb8-a003bc99f206","Type":"ContainerDied","Data":"4f3734123b7aae5b98f4eec189a605a2bd071aa52824a9e0f8a226611edbb3de"} Jul 23 21:06:09.324256 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.324220 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:09.456424 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.456339 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56nr6\" (UniqueName: \"kubernetes.io/projected/0f0a77f6-1431-4d4a-abe8-498eb204818b-kube-api-access-56nr6\") pod \"0f0a77f6-1431-4d4a-abe8-498eb204818b\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " Jul 23 21:06:09.456424 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.456393 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-bundle\") pod \"0f0a77f6-1431-4d4a-abe8-498eb204818b\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " Jul 23 21:06:09.456649 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.456479 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-util\") pod \"0f0a77f6-1431-4d4a-abe8-498eb204818b\" (UID: \"0f0a77f6-1431-4d4a-abe8-498eb204818b\") " Jul 23 21:06:09.456923 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.456896 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-bundle" (OuterVolumeSpecName: "bundle") pod "0f0a77f6-1431-4d4a-abe8-498eb204818b" (UID: "0f0a77f6-1431-4d4a-abe8-498eb204818b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:09.458527 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.458499 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0a77f6-1431-4d4a-abe8-498eb204818b-kube-api-access-56nr6" (OuterVolumeSpecName: "kube-api-access-56nr6") pod "0f0a77f6-1431-4d4a-abe8-498eb204818b" (UID: "0f0a77f6-1431-4d4a-abe8-498eb204818b"). InnerVolumeSpecName "kube-api-access-56nr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:09.547911 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.547867 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-util" (OuterVolumeSpecName: "util") pod "0f0a77f6-1431-4d4a-abe8-498eb204818b" (UID: "0f0a77f6-1431-4d4a-abe8-498eb204818b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:09.557681 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.557652 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:09.557681 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.557680 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f0a77f6-1431-4d4a-abe8-498eb204818b-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:09.557791 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:09.557689 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56nr6\" (UniqueName: \"kubernetes.io/projected/0f0a77f6-1431-4d4a-abe8-498eb204818b-kube-api-access-56nr6\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:10.193905 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.193876 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" Jul 23 21:06:10.194282 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.193904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x" event={"ID":"0f0a77f6-1431-4d4a-abe8-498eb204818b","Type":"ContainerDied","Data":"5d7220a528d3d3be9724ff6b7a489dc3084568c8a8c1d272216a160cafc20fb1"} Jul 23 21:06:10.194282 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.193955 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d7220a528d3d3be9724ff6b7a489dc3084568c8a8c1d272216a160cafc20fb1" Jul 23 21:06:10.359192 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.359167 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:10.360526 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.360498 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:10.464759 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.464720 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-bundle\") pod \"c26bd86a-52f8-47a1-ad05-e30418215620\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " Jul 23 21:06:10.464932 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.464794 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtxtj\" (UniqueName: \"kubernetes.io/projected/c26bd86a-52f8-47a1-ad05-e30418215620-kube-api-access-gtxtj\") pod \"c26bd86a-52f8-47a1-ad05-e30418215620\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " Jul 23 21:06:10.464932 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.464833 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-bundle\") pod \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " Jul 23 21:06:10.464932 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.464867 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-util\") pod \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " Jul 23 21:06:10.464932 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.464900 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-util\") pod \"c26bd86a-52f8-47a1-ad05-e30418215620\" (UID: \"c26bd86a-52f8-47a1-ad05-e30418215620\") " Jul 23 21:06:10.465139 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.464937 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbtfb\" (UniqueName: \"kubernetes.io/projected/c584ce0a-50c8-43d9-8eb8-a003bc99f206-kube-api-access-gbtfb\") pod \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\" (UID: \"c584ce0a-50c8-43d9-8eb8-a003bc99f206\") " Jul 23 21:06:10.465397 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.465363 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-bundle" (OuterVolumeSpecName: "bundle") pod "c26bd86a-52f8-47a1-ad05-e30418215620" (UID: "c26bd86a-52f8-47a1-ad05-e30418215620"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:10.465506 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.465417 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-bundle" (OuterVolumeSpecName: "bundle") pod "c584ce0a-50c8-43d9-8eb8-a003bc99f206" (UID: "c584ce0a-50c8-43d9-8eb8-a003bc99f206"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:10.467149 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.467118 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c26bd86a-52f8-47a1-ad05-e30418215620-kube-api-access-gtxtj" (OuterVolumeSpecName: "kube-api-access-gtxtj") pod "c26bd86a-52f8-47a1-ad05-e30418215620" (UID: "c26bd86a-52f8-47a1-ad05-e30418215620"). InnerVolumeSpecName "kube-api-access-gtxtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:10.467532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.467511 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c584ce0a-50c8-43d9-8eb8-a003bc99f206-kube-api-access-gbtfb" (OuterVolumeSpecName: "kube-api-access-gbtfb") pod "c584ce0a-50c8-43d9-8eb8-a003bc99f206" (UID: "c584ce0a-50c8-43d9-8eb8-a003bc99f206"). InnerVolumeSpecName "kube-api-access-gbtfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:10.470859 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.470834 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-util" (OuterVolumeSpecName: "util") pod "c26bd86a-52f8-47a1-ad05-e30418215620" (UID: "c26bd86a-52f8-47a1-ad05-e30418215620"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:10.471211 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.471189 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-util" (OuterVolumeSpecName: "util") pod "c584ce0a-50c8-43d9-8eb8-a003bc99f206" (UID: "c584ce0a-50c8-43d9-8eb8-a003bc99f206"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:10.566040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.565946 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbtfb\" (UniqueName: \"kubernetes.io/projected/c584ce0a-50c8-43d9-8eb8-a003bc99f206-kube-api-access-gbtfb\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:10.566040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.565979 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:10.566040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.565989 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtxtj\" (UniqueName: \"kubernetes.io/projected/c26bd86a-52f8-47a1-ad05-e30418215620-kube-api-access-gtxtj\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:10.566040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.565997 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-bundle\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:10.566040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.566006 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c584ce0a-50c8-43d9-8eb8-a003bc99f206-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:10.566040 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:10.566014 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c26bd86a-52f8-47a1-ad05-e30418215620-util\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:11.200403 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:11.200372 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" Jul 23 21:06:11.200403 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:11.200397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk" event={"ID":"c26bd86a-52f8-47a1-ad05-e30418215620","Type":"ContainerDied","Data":"8e08b29b2451cb9bbec9bc8ef2f3475edd3175d4016ca4a46474610c4d5388dc"} Jul 23 21:06:11.200910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:11.200460 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e08b29b2451cb9bbec9bc8ef2f3475edd3175d4016ca4a46474610c4d5388dc" Jul 23 21:06:11.208743 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:11.207037 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" event={"ID":"c584ce0a-50c8-43d9-8eb8-a003bc99f206","Type":"ContainerDied","Data":"167e1c11aab10335427eabc8fc18864701c1e847778aed1d5e0045c9c6ab5fc6"} Jul 23 21:06:11.208743 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:11.207137 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="167e1c11aab10335427eabc8fc18864701c1e847778aed1d5e0045c9c6ab5fc6" Jul 23 21:06:11.208743 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:11.207240 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44" Jul 23 21:06:20.027089 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027054 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf"] Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027387 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027398 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027407 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027414 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027420 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027425 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027432 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027439 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027446 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027451 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027458 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027463 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="util" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027469 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027474 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027480 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027484 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027491 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027497 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="pull" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027502 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027507 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027522 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027526 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027532 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="extract" Jul 23 21:06:20.027532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027537 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="extract" Jul 23 21:06:20.028243 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027628 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c26bd86a-52f8-47a1-ad05-e30418215620" containerName="extract" Jul 23 21:06:20.028243 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027637 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6c8b1fb-2ee0-4c64-a45c-6d19bd072426" containerName="extract" Jul 23 21:06:20.028243 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027644 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c584ce0a-50c8-43d9-8eb8-a003bc99f206" containerName="extract" Jul 23 21:06:20.028243 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.027651 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0a77f6-1431-4d4a-abe8-498eb204818b" containerName="extract" Jul 23 21:06:20.030731 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.030710 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:20.033747 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.033726 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-77df2\"" Jul 23 21:06:20.054830 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.054797 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf"] Jul 23 21:06:20.146261 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.146225 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cq26\" (UniqueName: \"kubernetes.io/projected/84d7bf43-fa45-4ff5-974d-01e58ea1b482-kube-api-access-6cq26\") pod \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" (UID: \"84d7bf43-fa45-4ff5-974d-01e58ea1b482\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:20.247313 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.247272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cq26\" (UniqueName: \"kubernetes.io/projected/84d7bf43-fa45-4ff5-974d-01e58ea1b482-kube-api-access-6cq26\") pod \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" (UID: \"84d7bf43-fa45-4ff5-974d-01e58ea1b482\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:20.257006 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.256974 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cq26\" (UniqueName: \"kubernetes.io/projected/84d7bf43-fa45-4ff5-974d-01e58ea1b482-kube-api-access-6cq26\") pod \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" (UID: \"84d7bf43-fa45-4ff5-974d-01e58ea1b482\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:20.341097 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.340998 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:20.486924 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:20.486898 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf"] Jul 23 21:06:20.489039 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:20.489009 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d7bf43_fa45_4ff5_974d_01e58ea1b482.slice/crio-296cbd15334b74ab683a1c8ba528fa105d7bbecbde9bd7f05d9ce05403fb7140 WatchSource:0}: Error finding container 296cbd15334b74ab683a1c8ba528fa105d7bbecbde9bd7f05d9ce05403fb7140: Status 404 returned error can't find the container with id 296cbd15334b74ab683a1c8ba528fa105d7bbecbde9bd7f05d9ce05403fb7140 Jul 23 21:06:21.260758 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:21.260723 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" event={"ID":"84d7bf43-fa45-4ff5-974d-01e58ea1b482","Type":"ContainerStarted","Data":"296cbd15334b74ab683a1c8ba528fa105d7bbecbde9bd7f05d9ce05403fb7140"} Jul 23 21:06:22.268283 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:22.268239 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" event={"ID":"84d7bf43-fa45-4ff5-974d-01e58ea1b482","Type":"ContainerStarted","Data":"2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4"} Jul 23 21:06:22.268709 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:22.268436 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:22.286367 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:22.286300 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" podStartSLOduration=0.699986042 podStartE2EDuration="2.286281665s" totalImagesPullingTime="1.586295623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:06:22.284643317 +0000 UTC m=+433.275189847" watchObservedRunningTime="2026-07-23 21:06:22.286281665 +0000 UTC m=+433.276828259" Jul 23 21:06:24.036704 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.036662 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-47gxn"] Jul 23 21:06:24.039243 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.039221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:24.042413 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.042387 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-blbh8\"" Jul 23 21:06:24.053434 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.053401 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-47gxn"] Jul 23 21:06:24.183370 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.183334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86svs\" (UniqueName: \"kubernetes.io/projected/ec467082-e718-4173-9b76-86851034ce3b-kube-api-access-86svs\") pod \"authorino-operator-75cc8d5cb7-47gxn\" (UID: \"ec467082-e718-4173-9b76-86851034ce3b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:24.284293 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.284257 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86svs\" (UniqueName: \"kubernetes.io/projected/ec467082-e718-4173-9b76-86851034ce3b-kube-api-access-86svs\") pod \"authorino-operator-75cc8d5cb7-47gxn\" (UID: \"ec467082-e718-4173-9b76-86851034ce3b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:24.292244 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.292170 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86svs\" (UniqueName: \"kubernetes.io/projected/ec467082-e718-4173-9b76-86851034ce3b-kube-api-access-86svs\") pod \"authorino-operator-75cc8d5cb7-47gxn\" (UID: \"ec467082-e718-4173-9b76-86851034ce3b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:24.350067 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.350023 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:24.500349 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:24.500266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-47gxn"] Jul 23 21:06:24.503294 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:24.503260 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec467082_e718_4173_9b76_86851034ce3b.slice/crio-30037759fef85c34fb5ea1b8b622062ae1040c76b1635916ce21afd8d29cd0f6 WatchSource:0}: Error finding container 30037759fef85c34fb5ea1b8b622062ae1040c76b1635916ce21afd8d29cd0f6: Status 404 returned error can't find the container with id 30037759fef85c34fb5ea1b8b622062ae1040c76b1635916ce21afd8d29cd0f6 Jul 23 21:06:25.287148 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:25.287108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" event={"ID":"ec467082-e718-4173-9b76-86851034ce3b","Type":"ContainerStarted","Data":"30037759fef85c34fb5ea1b8b622062ae1040c76b1635916ce21afd8d29cd0f6"} Jul 23 21:06:27.300651 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:27.300611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" event={"ID":"ec467082-e718-4173-9b76-86851034ce3b","Type":"ContainerStarted","Data":"9faaec293eb9c5d9bbb1def07a094b7e5c13979ca62c093c5d82410a9a47d7fa"} Jul 23 21:06:27.301047 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:27.300794 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:27.322634 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:27.321386 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" podStartSLOduration=1.025782989 podStartE2EDuration="3.321362336s" totalImagesPullingTime="2.295579347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:06:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:06:27.316774735 +0000 UTC m=+438.307321259" watchObservedRunningTime="2026-07-23 21:06:27.321362336 +0000 UTC m=+438.311908861" Jul 23 21:06:29.047700 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.047654 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs"] Jul 23 21:06:29.051072 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.051044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.053832 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.053727 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nw228\"" Jul 23 21:06:29.070199 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.070152 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs"] Jul 23 21:06:29.227930 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.227888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9zl\" (UniqueName: \"kubernetes.io/projected/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-kube-api-access-pj9zl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.228113 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.227947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.328767 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.328669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9zl\" (UniqueName: \"kubernetes.io/projected/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-kube-api-access-pj9zl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.328767 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.328744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.329156 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.329133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.339989 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.339952 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9zl\" (UniqueName: \"kubernetes.io/projected/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-kube-api-access-pj9zl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.362885 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.362845 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:29.508532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:29.508506 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs"] Jul 23 21:06:29.510401 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:29.510363 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b622d86_f262_4e04_bea6_5f5b8cfe62cc.slice/crio-881374422b9de6c3d37a3e834e77e9ed16266921ebde36c25c6c257bd43f39d6 WatchSource:0}: Error finding container 881374422b9de6c3d37a3e834e77e9ed16266921ebde36c25c6c257bd43f39d6: Status 404 returned error can't find the container with id 881374422b9de6c3d37a3e834e77e9ed16266921ebde36c25c6c257bd43f39d6 Jul 23 21:06:30.318948 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:30.318911 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" event={"ID":"3b622d86-f262-4e04-bea6-5f5b8cfe62cc","Type":"ContainerStarted","Data":"881374422b9de6c3d37a3e834e77e9ed16266921ebde36c25c6c257bd43f39d6"} Jul 23 21:06:33.276314 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:33.276229 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:35.356085 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:35.355989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" event={"ID":"3b622d86-f262-4e04-bea6-5f5b8cfe62cc","Type":"ContainerStarted","Data":"ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e"} Jul 23 21:06:35.356085 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:35.356053 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:35.375181 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:35.375103 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" podStartSLOduration=0.841188282 podStartE2EDuration="6.375084619s" totalImagesPullingTime="5.533896337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:06:35.373290115 +0000 UTC m=+446.363836652" watchObservedRunningTime="2026-07-23 21:06:35.375084619 +0000 UTC m=+446.365631141" Jul 23 21:06:38.308050 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:38.308018 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-47gxn" Jul 23 21:06:46.363452 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:46.363416 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:47.732233 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.732195 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6"] Jul 23 21:06:47.737003 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.736981 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:47.749378 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.749350 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6"] Jul 23 21:06:47.795736 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.795687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2664eee-7c9c-42eb-9738-c13fdafc9454-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:47.795918 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.795799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxd7z\" (UniqueName: \"kubernetes.io/projected/f2664eee-7c9c-42eb-9738-c13fdafc9454-kube-api-access-vxd7z\") pod \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:47.896894 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.896853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxd7z\" (UniqueName: \"kubernetes.io/projected/f2664eee-7c9c-42eb-9738-c13fdafc9454-kube-api-access-vxd7z\") pod \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:47.897094 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.896908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2664eee-7c9c-42eb-9738-c13fdafc9454-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:47.897292 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.897270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2664eee-7c9c-42eb-9738-c13fdafc9454-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:47.907768 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:47.907739 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxd7z\" (UniqueName: \"kubernetes.io/projected/f2664eee-7c9c-42eb-9738-c13fdafc9454-kube-api-access-vxd7z\") pod \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:48.022605 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.022462 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6"] Jul 23 21:06:48.022849 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.022830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:48.035705 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.035669 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6"] Jul 23 21:06:48.047619 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.047569 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs"] Jul 23 21:06:48.047958 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.047816 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" containerName="manager" containerID="cri-o://ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e" gracePeriod=2 Jul 23 21:06:48.062608 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.062574 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs"] Jul 23 21:06:48.070165 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.070130 2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-b6scs\": pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.087482 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.087444 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp"] Jul 23 21:06:48.088164 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.087853 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" containerName="manager" Jul 23 21:06:48.088164 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.087867 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" containerName="manager" Jul 23 21:06:48.088164 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.087938 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" containerName="manager" Jul 23 21:06:48.091219 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.091200 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf"] Jul 23 21:06:48.091343 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.091322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.091478 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.091447 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" containerName="manager" containerID="cri-o://2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4" gracePeriod=2 Jul 23 21:06:48.100078 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.100037 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf"] Jul 23 21:06:48.109837 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.108962 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp"] Jul 23 21:06:48.119487 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.119454 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x"] Jul 23 21:06:48.119873 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.119855 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" containerName="manager" Jul 23 21:06:48.119873 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.119871 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" containerName="manager" Jul 23 21:06:48.120048 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.119930 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" containerName="manager" Jul 23 21:06:48.123654 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.123427 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:06:48.140497 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.140454 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.145035 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.145007 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x"] Jul 23 21:06:48.159711 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.159660 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.161513 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.161484 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.200314 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.200288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvjpv\" (UniqueName: \"kubernetes.io/projected/c7cb4522-5a5a-4106-95f3-e3beb489223b-kube-api-access-pvjpv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-htpnp\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.200407 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.200347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks5nh\" (UniqueName: \"kubernetes.io/projected/123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f-kube-api-access-ks5nh\") pod \"limitador-operator-controller-manager-859d7b78bd-w9z8x\" (UID: \"123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:06:48.200407 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.200373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7cb4522-5a5a-4106-95f3-e3beb489223b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-htpnp\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.301329 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.301281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks5nh\" (UniqueName: \"kubernetes.io/projected/123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f-kube-api-access-ks5nh\") pod \"limitador-operator-controller-manager-859d7b78bd-w9z8x\" (UID: \"123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:06:48.301512 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.301338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7cb4522-5a5a-4106-95f3-e3beb489223b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-htpnp\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.301512 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.301443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvjpv\" (UniqueName: \"kubernetes.io/projected/c7cb4522-5a5a-4106-95f3-e3beb489223b-kube-api-access-pvjpv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-htpnp\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.301790 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.301765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7cb4522-5a5a-4106-95f3-e3beb489223b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-htpnp\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.312070 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.312038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks5nh\" (UniqueName: \"kubernetes.io/projected/123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f-kube-api-access-ks5nh\") pod \"limitador-operator-controller-manager-859d7b78bd-w9z8x\" (UID: \"123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:06:48.312070 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.312061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvjpv\" (UniqueName: \"kubernetes.io/projected/c7cb4522-5a5a-4106-95f3-e3beb489223b-kube-api-access-pvjpv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-htpnp\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.335368 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.335346 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.337519 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.337490 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.339268 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.339232 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.342226 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.342204 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.344321 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.344287 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.346146 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.346120 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.401925 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.401879 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj9zl\" (UniqueName: \"kubernetes.io/projected/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-kube-api-access-pj9zl\") pod \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " Jul 23 21:06:48.402099 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.402016 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cq26\" (UniqueName: \"kubernetes.io/projected/84d7bf43-fa45-4ff5-974d-01e58ea1b482-kube-api-access-6cq26\") pod \"84d7bf43-fa45-4ff5-974d-01e58ea1b482\" (UID: \"84d7bf43-fa45-4ff5-974d-01e58ea1b482\") " Jul 23 21:06:48.402099 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.402070 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-extensions-socket-volume\") pod \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\" (UID: \"3b622d86-f262-4e04-bea6-5f5b8cfe62cc\") " Jul 23 21:06:48.402532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.402510 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3b622d86-f262-4e04-bea6-5f5b8cfe62cc" (UID: "3b622d86-f262-4e04-bea6-5f5b8cfe62cc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:48.404118 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.404090 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d7bf43-fa45-4ff5-974d-01e58ea1b482-kube-api-access-6cq26" (OuterVolumeSpecName: "kube-api-access-6cq26") pod "84d7bf43-fa45-4ff5-974d-01e58ea1b482" (UID: "84d7bf43-fa45-4ff5-974d-01e58ea1b482"). InnerVolumeSpecName "kube-api-access-6cq26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:48.404224 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.404123 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-kube-api-access-pj9zl" (OuterVolumeSpecName: "kube-api-access-pj9zl") pod "3b622d86-f262-4e04-bea6-5f5b8cfe62cc" (UID: "3b622d86-f262-4e04-bea6-5f5b8cfe62cc"). InnerVolumeSpecName "kube-api-access-pj9zl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:48.429323 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.429289 2578 generic.go:356] "Generic (PLEG): container finished" podID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" containerID="ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e" exitCode=0 Jul 23 21:06:48.429513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.429333 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.429513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.429389 2578 scope.go:117] "RemoveContainer" containerID="ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e" Jul 23 21:06:48.431002 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.430974 2578 generic.go:356] "Generic (PLEG): container finished" podID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" containerID="2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4" exitCode=0 Jul 23 21:06:48.431113 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.431029 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.431509 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.431484 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.433212 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.433186 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.435918 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.435884 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.437967 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.437928 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.440283 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.440264 2578 scope.go:117] "RemoveContainer" containerID="ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e" Jul 23 21:06:48.440674 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.440648 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e\": container with ID starting with ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e not found: ID does not exist" containerID="ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e" Jul 23 21:06:48.440781 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.440689 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e"} err="failed to get container status \"ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e\": rpc error: code = NotFound desc = could not find container \"ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e\": container with ID starting with ae6373381035b842c264e3212611b138c1afaf0f411cc77e8101b3e6b3a1fa1e not found: ID does not exist" Jul 23 21:06:48.440781 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.440715 2578 scope.go:117] "RemoveContainer" containerID="2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4" Jul 23 21:06:48.450254 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.450225 2578 scope.go:117] "RemoveContainer" containerID="2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4" Jul 23 21:06:48.450571 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.450530 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4\": container with ID starting with 2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4 not found: ID does not exist" containerID="2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4" Jul 23 21:06:48.450646 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.450578 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4"} err="failed to get container status \"2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4\": rpc error: code = NotFound desc = could not find container \"2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4\": container with ID starting with 2132436acbd319f1366d5f0a090efbd17e080761c1c92eb92e8f88fad91bc2d4 not found: ID does not exist" Jul 23 21:06:48.454434 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.454400 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.457005 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.456982 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.459404 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.459382 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:48.461396 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:48.461375 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:48.489950 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.489901 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:48.496139 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.496117 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:06:48.502851 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.502823 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cq26\" (UniqueName: \"kubernetes.io/projected/84d7bf43-fa45-4ff5-974d-01e58ea1b482-kube-api-access-6cq26\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:48.502851 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.502850 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-extensions-socket-volume\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:48.502987 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.502859 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj9zl\" (UniqueName: \"kubernetes.io/projected/3b622d86-f262-4e04-bea6-5f5b8cfe62cc-kube-api-access-pj9zl\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:48.647129 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.647100 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp"] Jul 23 21:06:48.648965 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:48.648933 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7cb4522_5a5a_4106_95f3_e3beb489223b.slice/crio-eda025c3bb0031b7eef2e78a2385ac6eb44521cabf9bf7c7423e622a325b9368 WatchSource:0}: Error finding container eda025c3bb0031b7eef2e78a2385ac6eb44521cabf9bf7c7423e622a325b9368: Status 404 returned error can't find the container with id eda025c3bb0031b7eef2e78a2385ac6eb44521cabf9bf7c7423e622a325b9368 Jul 23 21:06:48.670879 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:48.670851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x"] Jul 23 21:06:48.672724 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:06:48.672700 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123e7ca0_3247_4b1f_8dd0_5cb4f36ff89f.slice/crio-559f72cbc68f8affcb9be304fe6afb41f05b8d0886cb9a483b2a3cda84095746 WatchSource:0}: Error finding container 559f72cbc68f8affcb9be304fe6afb41f05b8d0886cb9a483b2a3cda84095746: Status 404 returned error can't find the container with id 559f72cbc68f8affcb9be304fe6afb41f05b8d0886cb9a483b2a3cda84095746 Jul 23 21:06:49.439174 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.439133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" event={"ID":"123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f","Type":"ContainerStarted","Data":"623a29215294813539d92c452a845bdfc1638c8d1c8722f879bf2e463e20fc0c"} Jul 23 21:06:49.439707 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.439200 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" event={"ID":"123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f","Type":"ContainerStarted","Data":"559f72cbc68f8affcb9be304fe6afb41f05b8d0886cb9a483b2a3cda84095746"} Jul 23 21:06:49.439707 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.439312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:06:49.441457 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:49.441426 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:49.441657 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.441636 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" event={"ID":"c7cb4522-5a5a-4106-95f3-e3beb489223b","Type":"ContainerStarted","Data":"577e282f3e714ad1b4e2f3fa97743160149f98e88d98f87bfc0894ecc2bf5cb6"} Jul 23 21:06:49.441703 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.441682 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" event={"ID":"c7cb4522-5a5a-4106-95f3-e3beb489223b","Type":"ContainerStarted","Data":"eda025c3bb0031b7eef2e78a2385ac6eb44521cabf9bf7c7423e622a325b9368"} Jul 23 21:06:49.441734 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.441712 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:06:49.459663 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:49.459619 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:49.459837 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.459759 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" podStartSLOduration=1.4597426439999999 podStartE2EDuration="1.459742644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:06:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:06:49.457664346 +0000 UTC m=+460.448210875" watchObservedRunningTime="2026-07-23 21:06:49.459742644 +0000 UTC m=+460.450289166" Jul 23 21:06:49.480863 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:49.480808 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" podStartSLOduration=1.4807891579999999 podStartE2EDuration="1.480789158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:06:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:06:49.479435408 +0000 UTC m=+460.469981958" watchObservedRunningTime="2026-07-23 21:06:49.480789158 +0000 UTC m=+460.471335681" Jul 23 21:06:49.481355 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:49.481295 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:49.483025 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:49.483000 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:50.008524 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:50.008468 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-b6scs\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6scs" Jul 23 21:06:50.009703 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:50.009677 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b622d86-f262-4e04-bea6-5f5b8cfe62cc" path="/var/lib/kubelet/pods/3b622d86-f262-4e04-bea6-5f5b8cfe62cc/volumes" Jul 23 21:06:50.010026 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:50.010013 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" path="/var/lib/kubelet/pods/84d7bf43-fa45-4ff5-974d-01e58ea1b482/volumes" Jul 23 21:06:50.010675 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:50.010652 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rdtxf\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="84d7bf43-fa45-4ff5-974d-01e58ea1b482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rdtxf" Jul 23 21:06:50.569698 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:50.569656 2578 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-j72j6_kuadrant-system_f2664eee-7c9c-42eb-9738-c13fdafc9454_0(c67c712537b06460a018abe5c1aa255dcc34f5f8b0e21df8007b2a29adb594e8): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-j72j6 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 23 21:06:50.570006 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:50.569732 2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-j72j6_kuadrant-system_f2664eee-7c9c-42eb-9738-c13fdafc9454_0(c67c712537b06460a018abe5c1aa255dcc34f5f8b0e21df8007b2a29adb594e8): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-j72j6 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:51.453523 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.453490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:51.461974 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.461945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:51.465645 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:51.465609 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="f2664eee-7c9c-42eb-9738-c13fdafc9454" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:51.528220 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.528175 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxd7z\" (UniqueName: \"kubernetes.io/projected/f2664eee-7c9c-42eb-9738-c13fdafc9454-kube-api-access-vxd7z\") pod \"f2664eee-7c9c-42eb-9738-c13fdafc9454\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " Jul 23 21:06:51.528388 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.528231 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2664eee-7c9c-42eb-9738-c13fdafc9454-extensions-socket-volume\") pod \"f2664eee-7c9c-42eb-9738-c13fdafc9454\" (UID: \"f2664eee-7c9c-42eb-9738-c13fdafc9454\") " Jul 23 21:06:51.528524 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.528498 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2664eee-7c9c-42eb-9738-c13fdafc9454-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f2664eee-7c9c-42eb-9738-c13fdafc9454" (UID: "f2664eee-7c9c-42eb-9738-c13fdafc9454"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:06:51.530385 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.530364 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2664eee-7c9c-42eb-9738-c13fdafc9454-kube-api-access-vxd7z" (OuterVolumeSpecName: "kube-api-access-vxd7z") pod "f2664eee-7c9c-42eb-9738-c13fdafc9454" (UID: "f2664eee-7c9c-42eb-9738-c13fdafc9454"). InnerVolumeSpecName "kube-api-access-vxd7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:06:51.629504 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.629463 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxd7z\" (UniqueName: \"kubernetes.io/projected/f2664eee-7c9c-42eb-9738-c13fdafc9454-kube-api-access-vxd7z\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:51.629504 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:51.629498 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2664eee-7c9c-42eb-9738-c13fdafc9454-extensions-socket-volume\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:06:52.009513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:52.009476 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2664eee-7c9c-42eb-9738-c13fdafc9454" path="/var/lib/kubelet/pods/f2664eee-7c9c-42eb-9738-c13fdafc9454/volumes" Jul 23 21:06:52.458623 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:06:52.458588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:06:52.462709 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:06:52.462675 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-j72j6\" is forbidden: User \"system:node:ip-10-0-129-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-43.ec2.internal' and this object" podUID="f2664eee-7c9c-42eb-9738-c13fdafc9454" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j72j6" Jul 23 21:07:00.450216 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:00.450176 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:07:00.450615 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:00.450274 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9z8x" Jul 23 21:07:17.377313 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.377272 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp"] Jul 23 21:07:17.377806 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.377564 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" podUID="c7cb4522-5a5a-4106-95f3-e3beb489223b" containerName="manager" containerID="cri-o://577e282f3e714ad1b4e2f3fa97743160149f98e88d98f87bfc0894ecc2bf5cb6" gracePeriod=10 Jul 23 21:07:17.584587 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.584529 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx"] Jul 23 21:07:17.586927 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.586909 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.598839 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.598669 2578 generic.go:356] "Generic (PLEG): container finished" podID="c7cb4522-5a5a-4106-95f3-e3beb489223b" containerID="577e282f3e714ad1b4e2f3fa97743160149f98e88d98f87bfc0894ecc2bf5cb6" exitCode=0 Jul 23 21:07:17.598839 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.598797 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" event={"ID":"c7cb4522-5a5a-4106-95f3-e3beb489223b","Type":"ContainerDied","Data":"577e282f3e714ad1b4e2f3fa97743160149f98e88d98f87bfc0894ecc2bf5cb6"} Jul 23 21:07:17.600384 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.600364 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx"] Jul 23 21:07:17.642251 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.642222 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:07:17.743365 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.743326 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvjpv\" (UniqueName: \"kubernetes.io/projected/c7cb4522-5a5a-4106-95f3-e3beb489223b-kube-api-access-pvjpv\") pod \"c7cb4522-5a5a-4106-95f3-e3beb489223b\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " Jul 23 21:07:17.743534 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.743398 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7cb4522-5a5a-4106-95f3-e3beb489223b-extensions-socket-volume\") pod \"c7cb4522-5a5a-4106-95f3-e3beb489223b\" (UID: \"c7cb4522-5a5a-4106-95f3-e3beb489223b\") " Jul 23 21:07:17.743534 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.743523 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/087e2522-88bb-4b28-a4f4-f9612f299743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pdx\" (UID: \"087e2522-88bb-4b28-a4f4-f9612f299743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.743696 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.743599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlpz4\" (UniqueName: \"kubernetes.io/projected/087e2522-88bb-4b28-a4f4-f9612f299743-kube-api-access-xlpz4\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pdx\" (UID: \"087e2522-88bb-4b28-a4f4-f9612f299743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.743879 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.743846 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cb4522-5a5a-4106-95f3-e3beb489223b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c7cb4522-5a5a-4106-95f3-e3beb489223b" (UID: "c7cb4522-5a5a-4106-95f3-e3beb489223b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:07:17.745529 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.745509 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7cb4522-5a5a-4106-95f3-e3beb489223b-kube-api-access-pvjpv" (OuterVolumeSpecName: "kube-api-access-pvjpv") pod "c7cb4522-5a5a-4106-95f3-e3beb489223b" (UID: "c7cb4522-5a5a-4106-95f3-e3beb489223b"). InnerVolumeSpecName "kube-api-access-pvjpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:07:17.844519 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.844469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlpz4\" (UniqueName: \"kubernetes.io/projected/087e2522-88bb-4b28-a4f4-f9612f299743-kube-api-access-xlpz4\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pdx\" (UID: \"087e2522-88bb-4b28-a4f4-f9612f299743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.844735 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.844579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/087e2522-88bb-4b28-a4f4-f9612f299743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pdx\" (UID: \"087e2522-88bb-4b28-a4f4-f9612f299743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.844735 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.844619 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvjpv\" (UniqueName: \"kubernetes.io/projected/c7cb4522-5a5a-4106-95f3-e3beb489223b-kube-api-access-pvjpv\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:07:17.844735 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.844629 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7cb4522-5a5a-4106-95f3-e3beb489223b-extensions-socket-volume\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:07:17.844944 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.844928 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/087e2522-88bb-4b28-a4f4-f9612f299743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pdx\" (UID: \"087e2522-88bb-4b28-a4f4-f9612f299743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.853033 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.852993 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlpz4\" (UniqueName: \"kubernetes.io/projected/087e2522-88bb-4b28-a4f4-f9612f299743-kube-api-access-xlpz4\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pdx\" (UID: \"087e2522-88bb-4b28-a4f4-f9612f299743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:17.899069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:17.898979 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:18.075623 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.075596 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx"] Jul 23 21:07:18.077753 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:07:18.077727 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod087e2522_88bb_4b28_a4f4_f9612f299743.slice/crio-0c907dff862dbab65287cf3d4f68a1f459d561857b405da33e1bc529c9e6d994 WatchSource:0}: Error finding container 0c907dff862dbab65287cf3d4f68a1f459d561857b405da33e1bc529c9e6d994: Status 404 returned error can't find the container with id 0c907dff862dbab65287cf3d4f68a1f459d561857b405da33e1bc529c9e6d994 Jul 23 21:07:18.606731 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.606697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" event={"ID":"087e2522-88bb-4b28-a4f4-f9612f299743","Type":"ContainerStarted","Data":"5562d5599e3c42b8416eb05745ce39f04780157391c9d7c068766139d3c605b1"} Jul 23 21:07:18.607183 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.606762 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:18.607183 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.606783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" event={"ID":"087e2522-88bb-4b28-a4f4-f9612f299743","Type":"ContainerStarted","Data":"0c907dff862dbab65287cf3d4f68a1f459d561857b405da33e1bc529c9e6d994"} Jul 23 21:07:18.608094 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.608067 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" Jul 23 21:07:18.608094 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.608091 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp" event={"ID":"c7cb4522-5a5a-4106-95f3-e3beb489223b","Type":"ContainerDied","Data":"eda025c3bb0031b7eef2e78a2385ac6eb44521cabf9bf7c7423e622a325b9368"} Jul 23 21:07:18.608264 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.608139 2578 scope.go:117] "RemoveContainer" containerID="577e282f3e714ad1b4e2f3fa97743160149f98e88d98f87bfc0894ecc2bf5cb6" Jul 23 21:07:18.629104 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.629049 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" podStartSLOduration=1.6290279619999999 podStartE2EDuration="1.629027962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:07:18.626841414 +0000 UTC m=+489.617387938" watchObservedRunningTime="2026-07-23 21:07:18.629027962 +0000 UTC m=+489.619574485" Jul 23 21:07:18.641139 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.641101 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp"] Jul 23 21:07:18.647391 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:18.647357 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-htpnp"] Jul 23 21:07:20.013629 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:20.013592 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7cb4522-5a5a-4106-95f3-e3beb489223b" path="/var/lib/kubelet/pods/c7cb4522-5a5a-4106-95f3-e3beb489223b/volumes" Jul 23 21:07:29.617637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:29.617605 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pdx" Jul 23 21:07:33.674438 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.674400 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk"] Jul 23 21:07:33.674860 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.674810 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7cb4522-5a5a-4106-95f3-e3beb489223b" containerName="manager" Jul 23 21:07:33.674860 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.674823 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7cb4522-5a5a-4106-95f3-e3beb489223b" containerName="manager" Jul 23 21:07:33.674934 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.674881 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cb4522-5a5a-4106-95f3-e3beb489223b" containerName="manager" Jul 23 21:07:33.676974 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.676956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.679308 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.679283 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wnfwb\"" Jul 23 21:07:33.687977 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.687944 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk"] Jul 23 21:07:33.783535 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783535 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783535 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783574 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w94kr\" (UniqueName: \"kubernetes.io/projected/5df8fa90-0494-435a-bc3c-b2b603135e99-kube-api-access-w94kr\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df8fa90-0494-435a-bc3c-b2b603135e99-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.783844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.783792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885149 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885327 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w94kr\" (UniqueName: \"kubernetes.io/projected/5df8fa90-0494-435a-bc3c-b2b603135e99-kube-api-access-w94kr\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885327 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885327 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885450 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885450 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885584 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df8fa90-0494-435a-bc3c-b2b603135e99-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885584 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885702 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885765 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.885922 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885899 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.886000 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.885948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.886166 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.886143 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.886349 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.886325 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df8fa90-0494-435a-bc3c-b2b603135e99-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.888061 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.888036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.888266 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.888248 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.893601 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.893567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df8fa90-0494-435a-bc3c-b2b603135e99-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.893729 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.893665 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w94kr\" (UniqueName: \"kubernetes.io/projected/5df8fa90-0494-435a-bc3c-b2b603135e99-kube-api-access-w94kr\") pod \"maas-default-gateway-openshift-default-56654dcf7-tbbsk\" (UID: \"5df8fa90-0494-435a-bc3c-b2b603135e99\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:33.988761 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:33.988657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:34.136207 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.136177 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk"] Jul 23 21:07:34.138196 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:07:34.138154 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df8fa90_0494_435a_bc3c_b2b603135e99.slice/crio-83613f12356661cadf2db2595ab1efc590169319038f5f44309470f635732795 WatchSource:0}: Error finding container 83613f12356661cadf2db2595ab1efc590169319038f5f44309470f635732795: Status 404 returned error can't find the container with id 83613f12356661cadf2db2595ab1efc590169319038f5f44309470f635732795 Jul 23 21:07:34.140430 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.140399 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:07:34.140510 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.140462 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:07:34.140510 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.140495 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:07:34.697917 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.697874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" event={"ID":"5df8fa90-0494-435a-bc3c-b2b603135e99","Type":"ContainerStarted","Data":"1184d5b88cb6af737e049249a917654c52f3ac3431bc1314a47444862ea6198b"} Jul 23 21:07:34.698330 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.697943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" event={"ID":"5df8fa90-0494-435a-bc3c-b2b603135e99","Type":"ContainerStarted","Data":"83613f12356661cadf2db2595ab1efc590169319038f5f44309470f635732795"} Jul 23 21:07:34.718523 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.718469 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" podStartSLOduration=1.718453782 podStartE2EDuration="1.718453782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:07:34.717133081 +0000 UTC m=+505.707679606" watchObservedRunningTime="2026-07-23 21:07:34.718453782 +0000 UTC m=+505.709000309" Jul 23 21:07:34.989069 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:34.988972 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:35.995176 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:35.995141 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:36.708119 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:36.708086 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:36.709173 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:36.709149 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:07:37.065242 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.065129 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:37.068111 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.068081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.070276 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.070246 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 21:07:37.070400 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.070291 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9p2v6\"" Jul 23 21:07:37.075103 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.075075 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:37.113855 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.113809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d05539c-a10c-4893-8557-7538c4149271-config-file\") pod \"limitador-limitador-64c94fd64f-smc4q\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.114044 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.113877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m957\" (UniqueName: \"kubernetes.io/projected/7d05539c-a10c-4893-8557-7538c4149271-kube-api-access-5m957\") pod \"limitador-limitador-64c94fd64f-smc4q\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.169904 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.169869 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:37.215149 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.215114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d05539c-a10c-4893-8557-7538c4149271-config-file\") pod \"limitador-limitador-64c94fd64f-smc4q\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.215324 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.215171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m957\" (UniqueName: \"kubernetes.io/projected/7d05539c-a10c-4893-8557-7538c4149271-kube-api-access-5m957\") pod \"limitador-limitador-64c94fd64f-smc4q\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.215860 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.215838 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d05539c-a10c-4893-8557-7538c4149271-config-file\") pod \"limitador-limitador-64c94fd64f-smc4q\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.223621 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.223587 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m957\" (UniqueName: \"kubernetes.io/projected/7d05539c-a10c-4893-8557-7538c4149271-kube-api-access-5m957\") pod \"limitador-limitador-64c94fd64f-smc4q\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.380611 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.380476 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:37.534507 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.534473 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:37.537346 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:07:37.537299 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d05539c_a10c_4893_8557_7538c4149271.slice/crio-1d80849f89180092bcc3421ab5c5588e351d2f237add713637d5baa77cab735b WatchSource:0}: Error finding container 1d80849f89180092bcc3421ab5c5588e351d2f237add713637d5baa77cab735b: Status 404 returned error can't find the container with id 1d80849f89180092bcc3421ab5c5588e351d2f237add713637d5baa77cab735b Jul 23 21:07:37.715046 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:37.715011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" event={"ID":"7d05539c-a10c-4893-8557-7538c4149271","Type":"ContainerStarted","Data":"1d80849f89180092bcc3421ab5c5588e351d2f237add713637d5baa77cab735b"} Jul 23 21:07:38.083846 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.083763 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-ctbgq"] Jul 23 21:07:38.086890 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.086868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:07:38.089324 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.089296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xr9t7\"" Jul 23 21:07:38.097023 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.096991 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ctbgq"] Jul 23 21:07:38.121930 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.121884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zwk7\" (UniqueName: \"kubernetes.io/projected/dd562bb7-17fa-4eed-82ee-4efe92bb414e-kube-api-access-8zwk7\") pod \"authorino-75d556bd77-ctbgq\" (UID: \"dd562bb7-17fa-4eed-82ee-4efe92bb414e\") " pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:07:38.222960 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.222922 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zwk7\" (UniqueName: \"kubernetes.io/projected/dd562bb7-17fa-4eed-82ee-4efe92bb414e-kube-api-access-8zwk7\") pod \"authorino-75d556bd77-ctbgq\" (UID: \"dd562bb7-17fa-4eed-82ee-4efe92bb414e\") " pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:07:38.233110 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.231687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zwk7\" (UniqueName: \"kubernetes.io/projected/dd562bb7-17fa-4eed-82ee-4efe92bb414e-kube-api-access-8zwk7\") pod \"authorino-75d556bd77-ctbgq\" (UID: \"dd562bb7-17fa-4eed-82ee-4efe92bb414e\") " pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:07:38.398452 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.398095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:07:38.581217 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.581182 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ctbgq"] Jul 23 21:07:38.583382 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:07:38.583346 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd562bb7_17fa_4eed_82ee_4efe92bb414e.slice/crio-fa45b9656b8b91dfbffd123364cc43107f923deb6439724f502a67096718defc WatchSource:0}: Error finding container fa45b9656b8b91dfbffd123364cc43107f923deb6439724f502a67096718defc: Status 404 returned error can't find the container with id fa45b9656b8b91dfbffd123364cc43107f923deb6439724f502a67096718defc Jul 23 21:07:38.724153 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:38.724113 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ctbgq" event={"ID":"dd562bb7-17fa-4eed-82ee-4efe92bb414e","Type":"ContainerStarted","Data":"fa45b9656b8b91dfbffd123364cc43107f923deb6439724f502a67096718defc"} Jul 23 21:07:40.742378 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:40.742329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" event={"ID":"7d05539c-a10c-4893-8557-7538c4149271","Type":"ContainerStarted","Data":"d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5"} Jul 23 21:07:40.743066 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:40.742420 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:40.759748 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:40.759652 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" podStartSLOduration=0.932117531 podStartE2EDuration="3.759626724s" totalImagesPullingTime="2.827509193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:07:40.756853731 +0000 UTC m=+511.747400255" watchObservedRunningTime="2026-07-23 21:07:40.759626724 +0000 UTC m=+511.750173247" Jul 23 21:07:42.756538 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:42.756451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ctbgq" event={"ID":"dd562bb7-17fa-4eed-82ee-4efe92bb414e","Type":"ContainerStarted","Data":"b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6"} Jul 23 21:07:42.773557 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:42.773483 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-ctbgq" podStartSLOduration=0.902292345 podStartE2EDuration="4.773462791s" totalImagesPullingTime="3.871170446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:07:42.771379525 +0000 UTC m=+513.761926049" watchObservedRunningTime="2026-07-23 21:07:42.773462791 +0000 UTC m=+513.764009312" Jul 23 21:07:51.749597 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:51.749533 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:52.015764 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.015645 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:52.016028 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.015970 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" podUID="7d05539c-a10c-4893-8557-7538c4149271" containerName="limitador" containerID="cri-o://d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5" gracePeriod=30 Jul 23 21:07:52.573562 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.573479 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:52.633096 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.633054 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d05539c-a10c-4893-8557-7538c4149271-config-file\") pod \"7d05539c-a10c-4893-8557-7538c4149271\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " Jul 23 21:07:52.633096 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.633102 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m957\" (UniqueName: \"kubernetes.io/projected/7d05539c-a10c-4893-8557-7538c4149271-kube-api-access-5m957\") pod \"7d05539c-a10c-4893-8557-7538c4149271\" (UID: \"7d05539c-a10c-4893-8557-7538c4149271\") " Jul 23 21:07:52.633468 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.633443 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d05539c-a10c-4893-8557-7538c4149271-config-file" (OuterVolumeSpecName: "config-file") pod "7d05539c-a10c-4893-8557-7538c4149271" (UID: "7d05539c-a10c-4893-8557-7538c4149271"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:07:52.635307 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.635280 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d05539c-a10c-4893-8557-7538c4149271-kube-api-access-5m957" (OuterVolumeSpecName: "kube-api-access-5m957") pod "7d05539c-a10c-4893-8557-7538c4149271" (UID: "7d05539c-a10c-4893-8557-7538c4149271"). InnerVolumeSpecName "kube-api-access-5m957". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:07:52.734751 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.734703 2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d05539c-a10c-4893-8557-7538c4149271-config-file\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:07:52.734751 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.734742 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m957\" (UniqueName: \"kubernetes.io/projected/7d05539c-a10c-4893-8557-7538c4149271-kube-api-access-5m957\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:07:52.819675 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.819637 2578 generic.go:356] "Generic (PLEG): container finished" podID="7d05539c-a10c-4893-8557-7538c4149271" containerID="d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5" exitCode=0 Jul 23 21:07:52.820158 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.819681 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" event={"ID":"7d05539c-a10c-4893-8557-7538c4149271","Type":"ContainerDied","Data":"d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5"} Jul 23 21:07:52.820158 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.819723 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" event={"ID":"7d05539c-a10c-4893-8557-7538c4149271","Type":"ContainerDied","Data":"1d80849f89180092bcc3421ab5c5588e351d2f237add713637d5baa77cab735b"} Jul 23 21:07:52.820158 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.819743 2578 scope.go:117] "RemoveContainer" containerID="d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5" Jul 23 21:07:52.820158 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.819684 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-smc4q" Jul 23 21:07:52.829620 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.829592 2578 scope.go:117] "RemoveContainer" containerID="d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5" Jul 23 21:07:52.829934 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:07:52.829913 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5\": container with ID starting with d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5 not found: ID does not exist" containerID="d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5" Jul 23 21:07:52.829987 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.829946 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5"} err="failed to get container status \"d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5\": rpc error: code = NotFound desc = could not find container \"d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5\": container with ID starting with d3757462cf00b283adad4fe1dd92072344969e8e2a3cfceb83d361adaa7779c5 not found: ID does not exist" Jul 23 21:07:52.844971 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.844936 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:52.849207 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:52.849180 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-smc4q"] Jul 23 21:07:54.009617 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:07:54.009586 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d05539c-a10c-4893-8557-7538c4149271" path="/var/lib/kubelet/pods/7d05539c-a10c-4893-8557-7538c4149271/volumes" Jul 23 21:08:10.756608 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.756570 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cdc98574f-gfjx6"] Jul 23 21:08:10.757033 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.756943 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d05539c-a10c-4893-8557-7538c4149271" containerName="limitador" Jul 23 21:08:10.757033 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.756956 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d05539c-a10c-4893-8557-7538c4149271" containerName="limitador" Jul 23 21:08:10.757033 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.757020 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d05539c-a10c-4893-8557-7538c4149271" containerName="limitador" Jul 23 21:08:10.836945 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.836904 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cdc98574f-gfjx6"] Jul 23 21:08:10.837114 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.837011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:10.839513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.839491 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 21:08:10.899014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.898977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdhqc\" (UniqueName: \"kubernetes.io/projected/3f5b633e-cf43-400d-8dc3-2e61557c020c-kube-api-access-hdhqc\") pod \"authorino-5cdc98574f-gfjx6\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:10.899205 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:10.899076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f5b633e-cf43-400d-8dc3-2e61557c020c-tls-cert\") pod \"authorino-5cdc98574f-gfjx6\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:11.006672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.006574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f5b633e-cf43-400d-8dc3-2e61557c020c-tls-cert\") pod \"authorino-5cdc98574f-gfjx6\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:11.006672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.006667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdhqc\" (UniqueName: \"kubernetes.io/projected/3f5b633e-cf43-400d-8dc3-2e61557c020c-kube-api-access-hdhqc\") pod \"authorino-5cdc98574f-gfjx6\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:11.009130 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.009104 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f5b633e-cf43-400d-8dc3-2e61557c020c-tls-cert\") pod \"authorino-5cdc98574f-gfjx6\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:11.015461 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.015431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdhqc\" (UniqueName: \"kubernetes.io/projected/3f5b633e-cf43-400d-8dc3-2e61557c020c-kube-api-access-hdhqc\") pod \"authorino-5cdc98574f-gfjx6\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:11.146524 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.146484 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:08:11.308645 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.308614 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cdc98574f-gfjx6"] Jul 23 21:08:11.310430 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:08:11.310405 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f5b633e_cf43_400d_8dc3_2e61557c020c.slice/crio-951cc97741398ecc3f30425c952bd91c2059917500134ddbbf41f3d9307eccde WatchSource:0}: Error finding container 951cc97741398ecc3f30425c952bd91c2059917500134ddbbf41f3d9307eccde: Status 404 returned error can't find the container with id 951cc97741398ecc3f30425c952bd91c2059917500134ddbbf41f3d9307eccde Jul 23 21:08:11.923211 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:11.923176 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" event={"ID":"3f5b633e-cf43-400d-8dc3-2e61557c020c","Type":"ContainerStarted","Data":"951cc97741398ecc3f30425c952bd91c2059917500134ddbbf41f3d9307eccde"} Jul 23 21:08:12.930156 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:12.930117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" event={"ID":"3f5b633e-cf43-400d-8dc3-2e61557c020c","Type":"ContainerStarted","Data":"5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069"} Jul 23 21:08:12.946329 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:12.946273 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" podStartSLOduration=2.353949961 podStartE2EDuration="2.94625612s" totalImagesPullingTime="592.306159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:08:12.943907771 +0000 UTC m=+543.934454310" watchObservedRunningTime="2026-07-23 21:08:12.94625612 +0000 UTC m=+543.936802684" Jul 23 21:08:12.971143 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:12.971107 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ctbgq"] Jul 23 21:08:12.971413 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:12.971380 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-ctbgq" podUID="dd562bb7-17fa-4eed-82ee-4efe92bb414e" containerName="authorino" containerID="cri-o://b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6" gracePeriod=30 Jul 23 21:08:13.237463 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.237437 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:08:13.325529 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.325492 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zwk7\" (UniqueName: \"kubernetes.io/projected/dd562bb7-17fa-4eed-82ee-4efe92bb414e-kube-api-access-8zwk7\") pod \"dd562bb7-17fa-4eed-82ee-4efe92bb414e\" (UID: \"dd562bb7-17fa-4eed-82ee-4efe92bb414e\") " Jul 23 21:08:13.327704 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.327669 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd562bb7-17fa-4eed-82ee-4efe92bb414e-kube-api-access-8zwk7" (OuterVolumeSpecName: "kube-api-access-8zwk7") pod "dd562bb7-17fa-4eed-82ee-4efe92bb414e" (UID: "dd562bb7-17fa-4eed-82ee-4efe92bb414e"). InnerVolumeSpecName "kube-api-access-8zwk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:08:13.426733 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.426694 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zwk7\" (UniqueName: \"kubernetes.io/projected/dd562bb7-17fa-4eed-82ee-4efe92bb414e-kube-api-access-8zwk7\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:08:13.937062 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.937026 2578 generic.go:356] "Generic (PLEG): container finished" podID="dd562bb7-17fa-4eed-82ee-4efe92bb414e" containerID="b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6" exitCode=0 Jul 23 21:08:13.937062 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.937064 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ctbgq" Jul 23 21:08:13.937656 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.937113 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ctbgq" event={"ID":"dd562bb7-17fa-4eed-82ee-4efe92bb414e","Type":"ContainerDied","Data":"b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6"} Jul 23 21:08:13.937656 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.937153 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ctbgq" event={"ID":"dd562bb7-17fa-4eed-82ee-4efe92bb414e","Type":"ContainerDied","Data":"fa45b9656b8b91dfbffd123364cc43107f923deb6439724f502a67096718defc"} Jul 23 21:08:13.937656 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.937172 2578 scope.go:117] "RemoveContainer" containerID="b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6" Jul 23 21:08:13.946516 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.946492 2578 scope.go:117] "RemoveContainer" containerID="b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6" Jul 23 21:08:13.946850 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:08:13.946831 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6\": container with ID starting with b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6 not found: ID does not exist" containerID="b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6" Jul 23 21:08:13.946904 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.946859 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6"} err="failed to get container status \"b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6\": rpc error: code = NotFound desc = could not find container \"b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6\": container with ID starting with b555cc38df26657f263f76ca997177218491164c27e5ea00c7553f6f1138c7d6 not found: ID does not exist" Jul 23 21:08:13.957941 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.957912 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ctbgq"] Jul 23 21:08:13.960102 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:13.960067 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ctbgq"] Jul 23 21:08:14.009005 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:14.008971 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd562bb7-17fa-4eed-82ee-4efe92bb414e" path="/var/lib/kubelet/pods/dd562bb7-17fa-4eed-82ee-4efe92bb414e/volumes" Jul 23 21:08:53.865475 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.865428 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8srwc"] Jul 23 21:08:53.866065 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.866010 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd562bb7-17fa-4eed-82ee-4efe92bb414e" containerName="authorino" Jul 23 21:08:53.866065 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.866032 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd562bb7-17fa-4eed-82ee-4efe92bb414e" containerName="authorino" Jul 23 21:08:53.866205 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.866160 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd562bb7-17fa-4eed-82ee-4efe92bb414e" containerName="authorino" Jul 23 21:08:53.868735 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.868711 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:53.871184 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.871158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 21:08:53.871184 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.871173 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6t4j5\"" Jul 23 21:08:53.880212 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.880181 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8srwc"] Jul 23 21:08:53.916911 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.916876 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr"] Jul 23 21:08:53.919895 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.919874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:53.928226 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.928184 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr"] Jul 23 21:08:53.982448 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.982413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92884b49-a096-4614-a0fc-e46491081279-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8srwc\" (UID: \"92884b49-a096-4614-a0fc-e46491081279\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:53.982645 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.982464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31b8717c-2bff-40f2-b3f3-95051d91bb12-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-6nmvr\" (UID: \"31b8717c-2bff-40f2-b3f3-95051d91bb12\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:53.982645 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.982586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66q7t\" (UniqueName: \"kubernetes.io/projected/92884b49-a096-4614-a0fc-e46491081279-kube-api-access-66q7t\") pod \"payload-processing-5cc6cd444b-8srwc\" (UID: \"92884b49-a096-4614-a0fc-e46491081279\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:53.982645 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:53.982603 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl5jx\" (UniqueName: \"kubernetes.io/projected/31b8717c-2bff-40f2-b3f3-95051d91bb12-kube-api-access-dl5jx\") pod \"payload-pre-processing-687dbb6cf6-6nmvr\" (UID: \"31b8717c-2bff-40f2-b3f3-95051d91bb12\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:54.084014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.083974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66q7t\" (UniqueName: \"kubernetes.io/projected/92884b49-a096-4614-a0fc-e46491081279-kube-api-access-66q7t\") pod \"payload-processing-5cc6cd444b-8srwc\" (UID: \"92884b49-a096-4614-a0fc-e46491081279\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:54.084014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.084012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl5jx\" (UniqueName: \"kubernetes.io/projected/31b8717c-2bff-40f2-b3f3-95051d91bb12-kube-api-access-dl5jx\") pod \"payload-pre-processing-687dbb6cf6-6nmvr\" (UID: \"31b8717c-2bff-40f2-b3f3-95051d91bb12\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:54.084280 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.084052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92884b49-a096-4614-a0fc-e46491081279-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8srwc\" (UID: \"92884b49-a096-4614-a0fc-e46491081279\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:54.084280 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.084105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31b8717c-2bff-40f2-b3f3-95051d91bb12-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-6nmvr\" (UID: \"31b8717c-2bff-40f2-b3f3-95051d91bb12\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:54.084910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.084884 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31b8717c-2bff-40f2-b3f3-95051d91bb12-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-6nmvr\" (UID: \"31b8717c-2bff-40f2-b3f3-95051d91bb12\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:54.085010 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.084988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92884b49-a096-4614-a0fc-e46491081279-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8srwc\" (UID: \"92884b49-a096-4614-a0fc-e46491081279\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:54.092035 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.092001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl5jx\" (UniqueName: \"kubernetes.io/projected/31b8717c-2bff-40f2-b3f3-95051d91bb12-kube-api-access-dl5jx\") pod \"payload-pre-processing-687dbb6cf6-6nmvr\" (UID: \"31b8717c-2bff-40f2-b3f3-95051d91bb12\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:54.092178 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.092142 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66q7t\" (UniqueName: \"kubernetes.io/projected/92884b49-a096-4614-a0fc-e46491081279-kube-api-access-66q7t\") pod \"payload-processing-5cc6cd444b-8srwc\" (UID: \"92884b49-a096-4614-a0fc-e46491081279\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:54.180936 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.180887 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:54.232632 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.232400 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:54.334444 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.334393 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8srwc"] Jul 23 21:08:54.335255 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:08:54.335224 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92884b49_a096_4614_a0fc_e46491081279.slice/crio-1ef3498909b81284404bdc2d5e09b59849305d8d84a7b7eaeb6c7e4d6ff2a894 WatchSource:0}: Error finding container 1ef3498909b81284404bdc2d5e09b59849305d8d84a7b7eaeb6c7e4d6ff2a894: Status 404 returned error can't find the container with id 1ef3498909b81284404bdc2d5e09b59849305d8d84a7b7eaeb6c7e4d6ff2a894 Jul 23 21:08:54.392885 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:54.392841 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr"] Jul 23 21:08:54.395229 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:08:54.395197 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b8717c_2bff_40f2_b3f3_95051d91bb12.slice/crio-67dff949f46e3627a9f79973c6f14969b19d990630e60a562ffe2fe69bc023d8 WatchSource:0}: Error finding container 67dff949f46e3627a9f79973c6f14969b19d990630e60a562ffe2fe69bc023d8: Status 404 returned error can't find the container with id 67dff949f46e3627a9f79973c6f14969b19d990630e60a562ffe2fe69bc023d8 Jul 23 21:08:55.171786 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:55.171746 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" event={"ID":"31b8717c-2bff-40f2-b3f3-95051d91bb12","Type":"ContainerStarted","Data":"67dff949f46e3627a9f79973c6f14969b19d990630e60a562ffe2fe69bc023d8"} Jul 23 21:08:55.175593 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:55.175538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" event={"ID":"92884b49-a096-4614-a0fc-e46491081279","Type":"ContainerStarted","Data":"1ef3498909b81284404bdc2d5e09b59849305d8d84a7b7eaeb6c7e4d6ff2a894"} Jul 23 21:08:57.190955 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:57.190887 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" event={"ID":"31b8717c-2bff-40f2-b3f3-95051d91bb12","Type":"ContainerStarted","Data":"e01cd69c175479ffbf40178bfd3b7b9fe758713bbc8014326d79d7c59047e15c"} Jul 23 21:08:57.191393 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:57.190995 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:08:57.192343 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:57.192318 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" event={"ID":"92884b49-a096-4614-a0fc-e46491081279","Type":"ContainerStarted","Data":"529b1984a25a783172e25c3ba27f5fd48d059da95f5fa7c93652f6786a5eeb38"} Jul 23 21:08:57.192504 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:57.192490 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:08:57.207386 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:57.207324 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" podStartSLOduration=2.128267191 podStartE2EDuration="4.207308932s" totalImagesPullingTime="2.079041741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:08:57.204399289 +0000 UTC m=+588.194945811" watchObservedRunningTime="2026-07-23 21:08:57.207308932 +0000 UTC m=+588.197855455" Jul 23 21:08:57.223155 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:08:57.223071 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" podStartSLOduration=2.096108625 podStartE2EDuration="4.223044871s" totalImagesPullingTime="2.126936246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:08:57.219454806 +0000 UTC m=+588.210001335" watchObservedRunningTime="2026-07-23 21:08:57.223044871 +0000 UTC m=+588.213591397" Jul 23 21:09:02.223816 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:02.223778 2578 generic.go:356] "Generic (PLEG): container finished" podID="5df8fa90-0494-435a-bc3c-b2b603135e99" containerID="1184d5b88cb6af737e049249a917654c52f3ac3431bc1314a47444862ea6198b" exitCode=0 Jul 23 21:09:02.224344 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:02.223864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" event={"ID":"5df8fa90-0494-435a-bc3c-b2b603135e99","Type":"ContainerDied","Data":"1184d5b88cb6af737e049249a917654c52f3ac3431bc1314a47444862ea6198b"} Jul 23 21:09:02.224469 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:02.224365 2578 scope.go:117] "RemoveContainer" containerID="1184d5b88cb6af737e049249a917654c52f3ac3431bc1314a47444862ea6198b" Jul 23 21:09:02.225513 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:02.225484 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:09:02.225606 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:02.225589 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:09:02.225665 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:02.225653 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 21:09:03.236401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:03.236361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" event={"ID":"5df8fa90-0494-435a-bc3c-b2b603135e99","Type":"ContainerStarted","Data":"9c0e977380c4847b6050a76c78dac9ccea35da536db2900a93e7c4430a2a939e"} Jul 23 21:09:03.989204 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:03.989153 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:09:03.990371 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:03.990335 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 23 21:09:03.990506 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:03.990394 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" podUID="5df8fa90-0494-435a-bc3c-b2b603135e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 23 21:09:04.989972 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:04.989930 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 23 21:09:04.990402 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:04.989989 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" podUID="5df8fa90-0494-435a-bc3c-b2b603135e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 23 21:09:05.989577 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:05.989524 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 23 21:09:05.989764 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:05.989596 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" podUID="5df8fa90-0494-435a-bc3c-b2b603135e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 23 21:09:06.708150 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:06.708090 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:09:06.989974 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:06.989879 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 23 21:09:06.989974 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:06.989936 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" podUID="5df8fa90-0494-435a-bc3c-b2b603135e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 23 21:09:07.993475 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:07.993445 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:09:07.994654 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:07.994631 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tbbsk" Jul 23 21:09:08.199608 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:08.199574 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-8srwc" Jul 23 21:09:08.199775 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:08.199658 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-6nmvr" Jul 23 21:09:09.958890 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:09.958853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 21:09:09.961000 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:09.960976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 21:09:26.763468 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.763422 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp"] Jul 23 21:09:26.769426 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.769373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.771903 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.771838 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2l7z4\"" Jul 23 21:09:26.772075 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.771920 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 21:09:26.772075 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.771933 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 21:09:26.773251 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.773226 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 23 21:09:26.778170 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.777599 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp"] Jul 23 21:09:26.791998 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.791953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.792166 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.792026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b6a50b-297d-467c-8779-91cc51ee04d3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.792166 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.792070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2rs7\" (UniqueName: \"kubernetes.io/projected/06b6a50b-297d-467c-8779-91cc51ee04d3-kube-api-access-x2rs7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.792166 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.792142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.792310 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.792194 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.792310 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.792267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.892738 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.892700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2rs7\" (UniqueName: \"kubernetes.io/projected/06b6a50b-297d-467c-8779-91cc51ee04d3-kube-api-access-x2rs7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.892952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.892755 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.892952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.892802 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.892952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.892842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.892952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.892870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.892952 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.892914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b6a50b-297d-467c-8779-91cc51ee04d3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.893282 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.893259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.893356 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.893307 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.893356 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.893312 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.895264 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.895237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b6a50b-297d-467c-8779-91cc51ee04d3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.895463 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.895445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b6a50b-297d-467c-8779-91cc51ee04d3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:26.901281 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:26.901250 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2rs7\" (UniqueName: \"kubernetes.io/projected/06b6a50b-297d-467c-8779-91cc51ee04d3-kube-api-access-x2rs7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sz8kp\" (UID: \"06b6a50b-297d-467c-8779-91cc51ee04d3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:27.084252 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:27.084159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:27.236691 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:27.236656 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp"] Jul 23 21:09:27.239491 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:09:27.239459 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b6a50b_297d_467c_8779_91cc51ee04d3.slice/crio-d14c1eee32764cec80b9b87c986988edb466f51f437df2b07668ce471d57d6d6 WatchSource:0}: Error finding container d14c1eee32764cec80b9b87c986988edb466f51f437df2b07668ce471d57d6d6: Status 404 returned error can't find the container with id d14c1eee32764cec80b9b87c986988edb466f51f437df2b07668ce471d57d6d6 Jul 23 21:09:27.371657 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:27.371563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" event={"ID":"06b6a50b-297d-467c-8779-91cc51ee04d3","Type":"ContainerStarted","Data":"d14c1eee32764cec80b9b87c986988edb466f51f437df2b07668ce471d57d6d6"} Jul 23 21:09:29.386907 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:29.386863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" event={"ID":"06b6a50b-297d-467c-8779-91cc51ee04d3","Type":"ContainerStarted","Data":"fc37a7869445441a0e92a67d88b44b60b98ea9cababc83fc5ed2f9ace5e9df51"} Jul 23 21:09:29.404517 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:29.404462 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" podStartSLOduration=2.248395697 podStartE2EDuration="3.404444575s" totalImagesPullingTime="1.156048878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:09:29.402462069 +0000 UTC m=+620.393008605" watchObservedRunningTime="2026-07-23 21:09:29.404444575 +0000 UTC m=+620.394991150" Jul 23 21:09:37.084634 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:37.084529 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:37.084634 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:37.084616 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:37.098305 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:37.098277 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:37.442606 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:37.442578 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sz8kp" Jul 23 21:09:56.265468 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.265424 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4"] Jul 23 21:09:56.269301 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.269276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.271711 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.271688 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 21:09:56.279617 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.279583 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4"] Jul 23 21:09:56.370305 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.370263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.370493 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.370319 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.370493 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.370345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1643f98-69aa-4e08-af85-aef3d0d81d5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.370493 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.370373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.370493 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.370416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.370680 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.370486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66c5g\" (UniqueName: \"kubernetes.io/projected/c1643f98-69aa-4e08-af85-aef3d0d81d5f-kube-api-access-66c5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471339 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66c5g\" (UniqueName: \"kubernetes.io/projected/c1643f98-69aa-4e08-af85-aef3d0d81d5f-kube-api-access-66c5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471521 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471521 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471521 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1643f98-69aa-4e08-af85-aef3d0d81d5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471891 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.471959 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.471896 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.472062 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.472041 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.473997 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.473967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1643f98-69aa-4e08-af85-aef3d0d81d5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.474167 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.474150 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1643f98-69aa-4e08-af85-aef3d0d81d5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.480102 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.480079 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66c5g\" (UniqueName: \"kubernetes.io/projected/c1643f98-69aa-4e08-af85-aef3d0d81d5f-kube-api-access-66c5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2fzv4\" (UID: \"c1643f98-69aa-4e08-af85-aef3d0d81d5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.580577 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.580455 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:09:56.727211 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.727171 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4"] Jul 23 21:09:56.727496 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:09:56.727457 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1643f98_69aa_4e08_af85_aef3d0d81d5f.slice/crio-e9911e61c903ab41bea08af024c626fc39ecd2c33195633a669e7b0eb5b445ce WatchSource:0}: Error finding container e9911e61c903ab41bea08af024c626fc39ecd2c33195633a669e7b0eb5b445ce: Status 404 returned error can't find the container with id e9911e61c903ab41bea08af024c626fc39ecd2c33195633a669e7b0eb5b445ce Jul 23 21:09:56.729116 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:56.729091 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 21:09:57.551031 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:57.550988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" event={"ID":"c1643f98-69aa-4e08-af85-aef3d0d81d5f","Type":"ContainerStarted","Data":"fd7b596c885aa1223f58adacf516d79fa7dc97d576569fc2b79522a44d85b20d"} Jul 23 21:09:57.551450 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:57.551083 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" event={"ID":"c1643f98-69aa-4e08-af85-aef3d0d81d5f","Type":"ContainerStarted","Data":"e9911e61c903ab41bea08af024c626fc39ecd2c33195633a669e7b0eb5b445ce"} Jul 23 21:09:57.570405 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:09:57.570349 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" podStartSLOduration=1.358868363 podStartE2EDuration="1.570333023s" totalImagesPullingTime="211.46466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:09:57.567260395 +0000 UTC m=+648.557806919" watchObservedRunningTime="2026-07-23 21:09:57.570333023 +0000 UTC m=+648.560879580" Jul 23 21:10:06.580654 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:06.580606 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:10:06.581219 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:06.580684 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:10:06.593241 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:06.593209 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:10:06.609814 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:06.609783 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2fzv4" Jul 23 21:10:08.062628 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.062593 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts"] Jul 23 21:10:08.065477 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.065450 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.067841 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.067803 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 23 21:10:08.075761 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.075736 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts"] Jul 23 21:10:08.189516 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.189475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.189717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.189535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d3790bb-1078-4305-b035-6fb1fbab9a62-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.189717 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.189622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzv5s\" (UniqueName: \"kubernetes.io/projected/6d3790bb-1078-4305-b035-6fb1fbab9a62-kube-api-access-gzv5s\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.189794 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.189728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.189794 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.189768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.189868 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.189802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.290730 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.290686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d3790bb-1078-4305-b035-6fb1fbab9a62-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.290910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.290744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzv5s\" (UniqueName: \"kubernetes.io/projected/6d3790bb-1078-4305-b035-6fb1fbab9a62-kube-api-access-gzv5s\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.290910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.290784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.290910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.290803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.290910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.290836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.290910 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.290896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.291312 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.291287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.291402 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.291353 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.291402 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.291386 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.293063 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.293027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d3790bb-1078-4305-b035-6fb1fbab9a62-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.293479 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.293457 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d3790bb-1078-4305-b035-6fb1fbab9a62-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.298687 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.298660 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzv5s\" (UniqueName: \"kubernetes.io/projected/6d3790bb-1078-4305-b035-6fb1fbab9a62-kube-api-access-gzv5s\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x2hts\" (UID: \"6d3790bb-1078-4305-b035-6fb1fbab9a62\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.376891 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.376783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:08.524659 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.524625 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts"] Jul 23 21:10:08.525165 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:10:08.525138 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d3790bb_1078_4305_b035_6fb1fbab9a62.slice/crio-90dd760b82b26fdecce20ada10ec461782710440b5622d015679b5ad99d2ff36 WatchSource:0}: Error finding container 90dd760b82b26fdecce20ada10ec461782710440b5622d015679b5ad99d2ff36: Status 404 returned error can't find the container with id 90dd760b82b26fdecce20ada10ec461782710440b5622d015679b5ad99d2ff36 Jul 23 21:10:08.610084 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:08.610045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" event={"ID":"6d3790bb-1078-4305-b035-6fb1fbab9a62","Type":"ContainerStarted","Data":"90dd760b82b26fdecce20ada10ec461782710440b5622d015679b5ad99d2ff36"} Jul 23 21:10:09.618524 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:09.618483 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" event={"ID":"6d3790bb-1078-4305-b035-6fb1fbab9a62","Type":"ContainerStarted","Data":"22afae52ef0d5c51603db4eca7e3f05ad32e55fa054eaa1d5f960f06c9fbd11b"} Jul 23 21:10:09.635385 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:09.635330 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" podStartSLOduration=1.395649094 podStartE2EDuration="1.635312635s" totalImagesPullingTime="239.663541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:10:09.634164497 +0000 UTC m=+660.624711020" watchObservedRunningTime="2026-07-23 21:10:09.635312635 +0000 UTC m=+660.625859182" Jul 23 21:10:18.377357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:18.377313 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:18.377357 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:18.377367 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:18.390436 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:18.390405 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:10:18.687289 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:10:18.687256 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x2hts" Jul 23 21:11:07.635784 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.635697 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cdc98574f-gfjx6"] Jul 23 21:11:07.636339 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.635937 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" podUID="3f5b633e-cf43-400d-8dc3-2e61557c020c" containerName="authorino" containerID="cri-o://5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069" gracePeriod=30 Jul 23 21:11:07.905771 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.905746 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:11:07.945801 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.945761 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f5b633e-cf43-400d-8dc3-2e61557c020c-tls-cert\") pod \"3f5b633e-cf43-400d-8dc3-2e61557c020c\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " Jul 23 21:11:07.945986 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.945884 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdhqc\" (UniqueName: \"kubernetes.io/projected/3f5b633e-cf43-400d-8dc3-2e61557c020c-kube-api-access-hdhqc\") pod \"3f5b633e-cf43-400d-8dc3-2e61557c020c\" (UID: \"3f5b633e-cf43-400d-8dc3-2e61557c020c\") " Jul 23 21:11:07.947903 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.947869 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f5b633e-cf43-400d-8dc3-2e61557c020c-kube-api-access-hdhqc" (OuterVolumeSpecName: "kube-api-access-hdhqc") pod "3f5b633e-cf43-400d-8dc3-2e61557c020c" (UID: "3f5b633e-cf43-400d-8dc3-2e61557c020c"). InnerVolumeSpecName "kube-api-access-hdhqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:11:07.957281 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.957244 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5b633e-cf43-400d-8dc3-2e61557c020c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3f5b633e-cf43-400d-8dc3-2e61557c020c" (UID: "3f5b633e-cf43-400d-8dc3-2e61557c020c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:11:07.971713 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.970848 2578 generic.go:356] "Generic (PLEG): container finished" podID="3f5b633e-cf43-400d-8dc3-2e61557c020c" containerID="5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069" exitCode=0 Jul 23 21:11:07.971713 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.970985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" event={"ID":"3f5b633e-cf43-400d-8dc3-2e61557c020c","Type":"ContainerDied","Data":"5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069"} Jul 23 21:11:07.971713 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.971022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" event={"ID":"3f5b633e-cf43-400d-8dc3-2e61557c020c","Type":"ContainerDied","Data":"951cc97741398ecc3f30425c952bd91c2059917500134ddbbf41f3d9307eccde"} Jul 23 21:11:07.971713 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.971054 2578 scope.go:117] "RemoveContainer" containerID="5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069" Jul 23 21:11:07.971713 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.971316 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdc98574f-gfjx6" Jul 23 21:11:07.982708 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.982683 2578 scope.go:117] "RemoveContainer" containerID="5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069" Jul 23 21:11:07.982983 ip-10-0-129-43 kubenswrapper[2578]: E0723 21:11:07.982960 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069\": container with ID starting with 5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069 not found: ID does not exist" containerID="5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069" Jul 23 21:11:07.983050 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.982992 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069"} err="failed to get container status \"5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069\": rpc error: code = NotFound desc = could not find container \"5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069\": container with ID starting with 5e7388641d7aa29590e66538f8c4a19cce0b323f122163b7f6694e33c60f2069 not found: ID does not exist" Jul 23 21:11:07.998804 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:07.998774 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cdc98574f-gfjx6"] Jul 23 21:11:08.010059 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:08.010026 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cdc98574f-gfjx6"] Jul 23 21:11:08.046941 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:08.046906 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdhqc\" (UniqueName: \"kubernetes.io/projected/3f5b633e-cf43-400d-8dc3-2e61557c020c-kube-api-access-hdhqc\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:11:08.046941 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:08.046937 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f5b633e-cf43-400d-8dc3-2e61557c020c-tls-cert\") on node \"ip-10-0-129-43.ec2.internal\" DevicePath \"\"" Jul 23 21:11:10.009965 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:10.009928 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5b633e-cf43-400d-8dc3-2e61557c020c" path="/var/lib/kubelet/pods/3f5b633e-cf43-400d-8dc3-2e61557c020c/volumes" Jul 23 21:11:28.815271 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:28.815242 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-k9z7z_5edcce5e-7cfb-436b-b3f1-f80131b617d6/manager/0.log" Jul 23 21:11:28.927606 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:28.927572 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sw25l_736aa336-9275-43ae-955e-5076af0b5e3f/manager/0.log" Jul 23 21:11:29.159239 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:29.159162 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-644vr_d5cf93ef-3058-484b-a2da-58e05a718cca/server/0.log" Jul 23 21:11:29.264682 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:29.264643 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pck5h_8d78ac26-890f-49e1-92b9-9131d2bcc660/manager/0.log" Jul 23 21:11:29.486529 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:29.486498 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77944bd489-kl9vr_d6cbff37-d565-4cef-8524-2633993b8467/copy-manifests/0.log" Jul 23 21:11:29.493415 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:29.493393 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77944bd489-kl9vr_d6cbff37-d565-4cef-8524-2633993b8467/manager/0.log" Jul 23 21:11:30.789680 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:30.789651 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x_0f0a77f6-1431-4d4a-abe8-498eb204818b/util/0.log" Jul 23 21:11:30.796262 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:30.796233 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x_0f0a77f6-1431-4d4a-abe8-498eb204818b/pull/0.log" Jul 23 21:11:30.802225 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:30.802203 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x_0f0a77f6-1431-4d4a-abe8-498eb204818b/extract/0.log" Jul 23 21:11:30.910699 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:30.910668 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk_c26bd86a-52f8-47a1-ad05-e30418215620/util/0.log" Jul 23 21:11:30.916905 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:30.916886 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk_c26bd86a-52f8-47a1-ad05-e30418215620/pull/0.log" Jul 23 21:11:30.925103 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:30.925077 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk_c26bd86a-52f8-47a1-ad05-e30418215620/extract/0.log" Jul 23 21:11:31.030943 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.030910 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4_d6c8b1fb-2ee0-4c64-a45c-6d19bd072426/util/0.log" Jul 23 21:11:31.038844 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.038821 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4_d6c8b1fb-2ee0-4c64-a45c-6d19bd072426/pull/0.log" Jul 23 21:11:31.046015 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.045942 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4_d6c8b1fb-2ee0-4c64-a45c-6d19bd072426/extract/0.log" Jul 23 21:11:31.150745 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.150714 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44_c584ce0a-50c8-43d9-8eb8-a003bc99f206/util/0.log" Jul 23 21:11:31.157878 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.157850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44_c584ce0a-50c8-43d9-8eb8-a003bc99f206/pull/0.log" Jul 23 21:11:31.164452 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.164428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44_c584ce0a-50c8-43d9-8eb8-a003bc99f206/extract/0.log" Jul 23 21:11:31.395777 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.395696 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-47gxn_ec467082-e718-4173-9b76-86851034ce3b/manager/0.log" Jul 23 21:11:31.711902 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.711869 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pc5ph_4525cdd9-63e2-45a7-b9ba-e9373d6c0b60/registry-server/0.log" Jul 23 21:11:31.835275 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:31.835245 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s6pdx_087e2522-88bb-4b28-a4f4-f9612f299743/manager/0.log" Jul 23 21:11:32.053634 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:32.053540 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-w9z8x_123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f/manager/0.log" Jul 23 21:11:32.370579 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:32.370482 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45j9km_0a0d4e90-3322-48c9-a256-2c8057a43d89/istio-proxy/0.log" Jul 23 21:11:32.584370 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:32.584343 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cc7f4f888-kg4mf_77a9aa17-a753-4ae1-bbc9-511ad743d45d/kube-auth-proxy/0.log" Jul 23 21:11:32.801256 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:32.801226 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/1.log" Jul 23 21:11:32.911015 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:32.910985 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-6nmvr_31b8717c-2bff-40f2-b3f3-95051d91bb12/payload-pre-processing/0.log" Jul 23 21:11:33.019126 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:33.019096 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8srwc_92884b49-a096-4614-a0fc-e46491081279/payload-processing/0.log" Jul 23 21:11:33.130887 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:33.130793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d54bbc4cb-htvqd_e76bfe9e-26d6-4966-8069-cc0ffddcdef9/router/0.log" Jul 23 21:11:33.565660 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:33.565629 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-sz8kp_06b6a50b-297d-467c-8779-91cc51ee04d3/main/0.log" Jul 23 21:11:33.880980 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:33.880891 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2fzv4_c1643f98-69aa-4e08-af85-aef3d0d81d5f/main/0.log" Jul 23 21:11:33.990756 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:33.990729 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-x2hts_6d3790bb-1078-4305-b035-6fb1fbab9a62/main/0.log" Jul 23 21:11:38.993446 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.993404 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kxk68/must-gather-pnch6"] Jul 23 21:11:38.993902 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.993886 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5b633e-cf43-400d-8dc3-2e61557c020c" containerName="authorino" Jul 23 21:11:38.993961 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.993903 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5b633e-cf43-400d-8dc3-2e61557c020c" containerName="authorino" Jul 23 21:11:38.994014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.993977 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5b633e-cf43-400d-8dc3-2e61557c020c" containerName="authorino" Jul 23 21:11:38.997401 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.997379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.000014 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.999988 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kxk68\"/\"kube-root-ca.crt\"" Jul 23 21:11:39.000141 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:38.999988 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kxk68\"/\"openshift-service-ca.crt\"" Jul 23 21:11:39.000577 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.000562 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kxk68\"/\"default-dockercfg-fgvfl\"" Jul 23 21:11:39.007091 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.007063 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxk68/must-gather-pnch6"] Jul 23 21:11:39.140492 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.140455 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/da23aa91-b5b1-4ad4-abe5-391f84b092d2-must-gather-output\") pod \"must-gather-pnch6\" (UID: \"da23aa91-b5b1-4ad4-abe5-391f84b092d2\") " pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.140492 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.140495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wx9n\" (UniqueName: \"kubernetes.io/projected/da23aa91-b5b1-4ad4-abe5-391f84b092d2-kube-api-access-5wx9n\") pod \"must-gather-pnch6\" (UID: \"da23aa91-b5b1-4ad4-abe5-391f84b092d2\") " pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.241026 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.240979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/da23aa91-b5b1-4ad4-abe5-391f84b092d2-must-gather-output\") pod \"must-gather-pnch6\" (UID: \"da23aa91-b5b1-4ad4-abe5-391f84b092d2\") " pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.241026 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.241027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wx9n\" (UniqueName: \"kubernetes.io/projected/da23aa91-b5b1-4ad4-abe5-391f84b092d2-kube-api-access-5wx9n\") pod \"must-gather-pnch6\" (UID: \"da23aa91-b5b1-4ad4-abe5-391f84b092d2\") " pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.241351 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.241329 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/da23aa91-b5b1-4ad4-abe5-391f84b092d2-must-gather-output\") pod \"must-gather-pnch6\" (UID: \"da23aa91-b5b1-4ad4-abe5-391f84b092d2\") " pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.249893 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.249827 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wx9n\" (UniqueName: \"kubernetes.io/projected/da23aa91-b5b1-4ad4-abe5-391f84b092d2-kube-api-access-5wx9n\") pod \"must-gather-pnch6\" (UID: \"da23aa91-b5b1-4ad4-abe5-391f84b092d2\") " pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.309051 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.309011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxk68/must-gather-pnch6" Jul 23 21:11:39.455612 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:39.455162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxk68/must-gather-pnch6"] Jul 23 21:11:39.461660 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:11:39.461628 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda23aa91_b5b1_4ad4_abe5_391f84b092d2.slice/crio-f0adac7c2a0703278e43b575b4f92b74b9c0313f2ef6bd73af4069dd59f64e56 WatchSource:0}: Error finding container f0adac7c2a0703278e43b575b4f92b74b9c0313f2ef6bd73af4069dd59f64e56: Status 404 returned error can't find the container with id f0adac7c2a0703278e43b575b4f92b74b9c0313f2ef6bd73af4069dd59f64e56 Jul 23 21:11:40.156426 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:40.156392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxk68/must-gather-pnch6" event={"ID":"da23aa91-b5b1-4ad4-abe5-391f84b092d2","Type":"ContainerStarted","Data":"f0adac7c2a0703278e43b575b4f92b74b9c0313f2ef6bd73af4069dd59f64e56"} Jul 23 21:11:41.165714 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:41.165678 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxk68/must-gather-pnch6" event={"ID":"da23aa91-b5b1-4ad4-abe5-391f84b092d2","Type":"ContainerStarted","Data":"cd171202bf7f2758db61783e187d68101ecae885ca00cd78c2f08b54905f7bf1"} Jul 23 21:11:41.166139 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:41.165724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxk68/must-gather-pnch6" event={"ID":"da23aa91-b5b1-4ad4-abe5-391f84b092d2","Type":"ContainerStarted","Data":"127b0df3cb49c01fb46f1e1b0ff97c9cae113685fa03cc683f726510259039fc"} Jul 23 21:11:41.182508 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:41.182454 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kxk68/must-gather-pnch6" podStartSLOduration=2.334694227 podStartE2EDuration="3.182437923s" totalImagesPullingTime="847.743696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:11:41.180100694 +0000 UTC m=+752.170647218" watchObservedRunningTime="2026-07-23 21:11:41.182437923 +0000 UTC m=+752.172984446" Jul 23 21:11:41.978484 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:41.978447 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bs7mm_a6a4f6f0-c1d5-462f-ab87-eefaa2943c56/global-pull-secret-syncer/0.log" Jul 23 21:11:42.117638 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:42.117455 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-stjx7_0213c346-df99-4945-9f79-884622623238/konnectivity-agent/0.log" Jul 23 21:11:42.165640 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:42.165611 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-43.ec2.internal_07744b71299bb4e3a3b21c8d453b608e/haproxy/0.log" Jul 23 21:11:47.537693 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.537658 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x_0f0a77f6-1431-4d4a-abe8-498eb204818b/extract/0.log" Jul 23 21:11:47.578138 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.578054 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x_0f0a77f6-1431-4d4a-abe8-498eb204818b/util/0.log" Jul 23 21:11:47.610034 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.610007 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k8s8x_0f0a77f6-1431-4d4a-abe8-498eb204818b/pull/0.log" Jul 23 21:11:47.648106 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.648070 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk_c26bd86a-52f8-47a1-ad05-e30418215620/extract/0.log" Jul 23 21:11:47.672419 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.672393 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk_c26bd86a-52f8-47a1-ad05-e30418215620/util/0.log" Jul 23 21:11:47.700390 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.700306 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b2rxk_c26bd86a-52f8-47a1-ad05-e30418215620/pull/0.log" Jul 23 21:11:47.731091 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.731060 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4_d6c8b1fb-2ee0-4c64-a45c-6d19bd072426/extract/0.log" Jul 23 21:11:47.757617 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.757510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4_d6c8b1fb-2ee0-4c64-a45c-6d19bd072426/util/0.log" Jul 23 21:11:47.786334 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.786305 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m7md4_d6c8b1fb-2ee0-4c64-a45c-6d19bd072426/pull/0.log" Jul 23 21:11:47.817982 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.817901 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44_c584ce0a-50c8-43d9-8eb8-a003bc99f206/extract/0.log" Jul 23 21:11:47.843741 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.843712 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44_c584ce0a-50c8-43d9-8eb8-a003bc99f206/util/0.log" Jul 23 21:11:47.877497 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.877448 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvc44_c584ce0a-50c8-43d9-8eb8-a003bc99f206/pull/0.log" Jul 23 21:11:47.956559 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:47.956516 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-47gxn_ec467082-e718-4173-9b76-86851034ce3b/manager/0.log" Jul 23 21:11:48.110572 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:48.110462 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pc5ph_4525cdd9-63e2-45a7-b9ba-e9373d6c0b60/registry-server/0.log" Jul 23 21:11:48.186672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:48.186635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s6pdx_087e2522-88bb-4b28-a4f4-f9612f299743/manager/0.log" Jul 23 21:11:48.264052 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:48.263277 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-w9z8x_123e7ca0-3247-4b1f-8dd0-5cb4f36ff89f/manager/0.log" Jul 23 21:11:50.249651 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:50.249607 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-tbs7t_c6f338b8-691d-4985-9398-0452739f7aec/cluster-monitoring-operator/0.log" Jul 23 21:11:50.523437 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:50.523087 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjdst_3b2cf885-4c02-4b56-9da3-7ee4f97694e0/node-exporter/0.log" Jul 23 21:11:50.549056 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:50.549016 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjdst_3b2cf885-4c02-4b56-9da3-7ee4f97694e0/kube-rbac-proxy/0.log" Jul 23 21:11:50.572306 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:50.572274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjdst_3b2cf885-4c02-4b56-9da3-7ee4f97694e0/init-textfile/0.log" Jul 23 21:11:51.837967 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.837931 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr"] Jul 23 21:11:51.842321 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.842293 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.852435 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.852400 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr"] Jul 23 21:11:51.875297 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.875257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-proc\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.875610 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.875587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-lib-modules\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.875774 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.875756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-sys\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.875966 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.875948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f579r\" (UniqueName: \"kubernetes.io/projected/ad3a2803-773c-4352-9cfb-c36815318817-kube-api-access-f579r\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.876086 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.876072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-podres\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977592 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-proc\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977777 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-lib-modules\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977777 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-proc\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977777 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-sys\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977953 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977780 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-lib-modules\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977953 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f579r\" (UniqueName: \"kubernetes.io/projected/ad3a2803-773c-4352-9cfb-c36815318817-kube-api-access-f579r\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977953 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-podres\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977953 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-sys\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.977953 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.977941 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad3a2803-773c-4352-9cfb-c36815318817-podres\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:51.988241 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:51.988198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f579r\" (UniqueName: \"kubernetes.io/projected/ad3a2803-773c-4352-9cfb-c36815318817-kube-api-access-f579r\") pod \"perf-node-gather-daemonset-cq7kr\" (UID: \"ad3a2803-773c-4352-9cfb-c36815318817\") " pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:52.156718 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:52.156622 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:52.367205 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:52.367172 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr"] Jul 23 21:11:52.369670 ip-10-0-129-43 kubenswrapper[2578]: W0723 21:11:52.369636 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podad3a2803_773c_4352_9cfb_c36815318817.slice/crio-6848ccd0d6a678f18f892f15676ae139b9e700a3cdbb94415a9cc41c0d81dba8 WatchSource:0}: Error finding container 6848ccd0d6a678f18f892f15676ae139b9e700a3cdbb94415a9cc41c0d81dba8: Status 404 returned error can't find the container with id 6848ccd0d6a678f18f892f15676ae139b9e700a3cdbb94415a9cc41c0d81dba8 Jul 23 21:11:52.479101 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:52.479063 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-9tsh9_d06fea94-cf37-4c37-b739-350370a4699c/networking-console-plugin/0.log" Jul 23 21:11:53.089906 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.089867 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/1.log" Jul 23 21:11:53.096981 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.096951 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rwvmk_43b49123-f63a-45cf-a5e9-12aa4f0eff24/console-operator/2.log" Jul 23 21:11:53.274961 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.274923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" event={"ID":"ad3a2803-773c-4352-9cfb-c36815318817","Type":"ContainerStarted","Data":"df83b39462b89ae2aa78dd2b91183a85ed3eaf63f7a287facf0c27107a5c6ea1"} Jul 23 21:11:53.275173 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.274987 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" event={"ID":"ad3a2803-773c-4352-9cfb-c36815318817","Type":"ContainerStarted","Data":"6848ccd0d6a678f18f892f15676ae139b9e700a3cdbb94415a9cc41c0d81dba8"} Jul 23 21:11:53.275173 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.275119 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:11:53.293806 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.293745 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" podStartSLOduration=2.293727244 podStartE2EDuration="2.293727244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:11:53.291071065 +0000 UTC m=+764.281617639" watchObservedRunningTime="2026-07-23 21:11:53.293727244 +0000 UTC m=+764.284273766" Jul 23 21:11:53.628381 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:53.628350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-2lvbw_b7d53cd9-bf05-4224-baa3-012ba7c85d61/download-server/0.log" Jul 23 21:11:54.211784 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:54.211748 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-sx5mn_3114df95-e113-40a9-a92e-dbf21f91ff1b/volume-data-source-validator/0.log" Jul 23 21:11:55.310187 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:55.310150 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmf6p_ccf68472-95b3-4bf4-acf8-25ecb1726591/dns/0.log" Jul 23 21:11:55.338640 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:55.338608 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmf6p_ccf68472-95b3-4bf4-acf8-25ecb1726591/kube-rbac-proxy/0.log" Jul 23 21:11:55.437393 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:55.437362 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-72tl8_2b959712-8ea5-4d5c-97c8-c53de4c55ad9/dns-node-resolver/0.log" Jul 23 21:11:56.035262 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:56.035236 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d8j56_b22b0230-000b-4fd7-8596-b5832724cecb/node-ca/0.log" Jul 23 21:11:57.025344 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.025318 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45j9km_0a0d4e90-3322-48c9-a256-2c8057a43d89/istio-proxy/0.log" Jul 23 21:11:57.101723 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.101693 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cc7f4f888-kg4mf_77a9aa17-a753-4ae1-bbc9-511ad743d45d/kube-auth-proxy/0.log" Jul 23 21:11:57.156411 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.156378 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/1.log" Jul 23 21:11:57.156855 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.156823 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/0.log" Jul 23 21:11:57.183861 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.183832 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-6nmvr_31b8717c-2bff-40f2-b3f3-95051d91bb12/payload-pre-processing/0.log" Jul 23 21:11:57.209598 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.209540 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8srwc_92884b49-a096-4614-a0fc-e46491081279/payload-processing/0.log" Jul 23 21:11:57.236166 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.236137 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d54bbc4cb-htvqd_e76bfe9e-26d6-4966-8069-cc0ffddcdef9/router/0.log" Jul 23 21:11:57.875697 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:57.875665 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ffvc4_e724dd31-c672-47de-b929-30bb27fe25ef/serve-healthcheck-canary/0.log" Jul 23 21:11:58.413517 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:58.413482 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8g727_66d92b23-a2bf-442c-9986-0a10dcd10d7e/insights-operator/0.log" Jul 23 21:11:58.414114 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:58.414093 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8g727_66d92b23-a2bf-442c-9986-0a10dcd10d7e/insights-operator/1.log" Jul 23 21:11:58.439532 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:58.439499 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74pmk_cc6fd4de-2bbf-4c58-9af2-854e64e95d2f/kube-rbac-proxy/0.log" Jul 23 21:11:58.461348 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:58.461320 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74pmk_cc6fd4de-2bbf-4c58-9af2-854e64e95d2f/exporter/0.log" Jul 23 21:11:58.483486 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:58.483452 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74pmk_cc6fd4de-2bbf-4c58-9af2-854e64e95d2f/extractor/0.log" Jul 23 21:11:59.296571 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:11:59.296511 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kxk68/perf-node-gather-daemonset-cq7kr" Jul 23 21:12:00.857619 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:00.857502 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-k9z7z_5edcce5e-7cfb-436b-b3f1-f80131b617d6/manager/0.log" Jul 23 21:12:00.899364 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:00.899287 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sw25l_736aa336-9275-43ae-955e-5076af0b5e3f/manager/0.log" Jul 23 21:12:00.950637 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:00.950608 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-644vr_d5cf93ef-3058-484b-a2da-58e05a718cca/server/0.log" Jul 23 21:12:01.015252 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:01.015218 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pck5h_8d78ac26-890f-49e1-92b9-9131d2bcc660/manager/0.log" Jul 23 21:12:01.117274 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:01.117189 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77944bd489-kl9vr_d6cbff37-d565-4cef-8524-2633993b8467/manager/0.log" Jul 23 21:12:01.138814 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:01.138788 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77944bd489-kl9vr_d6cbff37-d565-4cef-8524-2633993b8467/copy-manifests/0.log" Jul 23 21:12:02.586153 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:02.586120 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cgnrq_515122d9-4572-49f3-a142-6fee1d0abc26/openshift-lws-operator/0.log" Jul 23 21:12:03.055129 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.055041 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45j9km_0a0d4e90-3322-48c9-a256-2c8057a43d89/istio-proxy/0.log" Jul 23 21:12:03.104849 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.104795 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cc7f4f888-kg4mf_77a9aa17-a753-4ae1-bbc9-511ad743d45d/kube-auth-proxy/0.log" Jul 23 21:12:03.150462 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.150430 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/1.log" Jul 23 21:12:03.150675 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.150537 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/0.log" Jul 23 21:12:03.162365 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.162311 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-6nmvr_31b8717c-2bff-40f2-b3f3-95051d91bb12/payload-pre-processing/0.log" Jul 23 21:12:03.175888 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.175858 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8srwc_92884b49-a096-4614-a0fc-e46491081279/payload-processing/0.log" Jul 23 21:12:03.193126 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:03.193093 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d54bbc4cb-htvqd_e76bfe9e-26d6-4966-8069-cc0ffddcdef9/router/0.log" Jul 23 21:12:04.310151 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.310067 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45j9km_0a0d4e90-3322-48c9-a256-2c8057a43d89/istio-proxy/0.log" Jul 23 21:12:04.359723 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.359627 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cc7f4f888-kg4mf_77a9aa17-a753-4ae1-bbc9-511ad743d45d/kube-auth-proxy/0.log" Jul 23 21:12:04.398002 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.397974 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/1.log" Jul 23 21:12:04.398205 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.398139 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tbbsk_5df8fa90-0494-435a-bc3c-b2b603135e99/istio-proxy/0.log" Jul 23 21:12:04.411162 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.411124 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-6nmvr_31b8717c-2bff-40f2-b3f3-95051d91bb12/payload-pre-processing/0.log" Jul 23 21:12:04.425127 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.425076 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8srwc_92884b49-a096-4614-a0fc-e46491081279/payload-processing/0.log" Jul 23 21:12:04.443146 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:04.443110 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d54bbc4cb-htvqd_e76bfe9e-26d6-4966-8069-cc0ffddcdef9/router/0.log" Jul 23 21:12:08.091132 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:08.091103 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-4b6xx_2565a914-16d9-4bf9-8adb-f07b5bcaaff4/kube-storage-version-migrator-operator/1.log" Jul 23 21:12:08.092114 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:08.092092 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-4b6xx_2565a914-16d9-4bf9-8adb-f07b5bcaaff4/kube-storage-version-migrator-operator/0.log" Jul 23 21:12:09.343116 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.343089 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/kube-multus-additional-cni-plugins/0.log" Jul 23 21:12:09.367672 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.367645 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/egress-router-binary-copy/0.log" Jul 23 21:12:09.390148 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.390116 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/cni-plugins/0.log" Jul 23 21:12:09.414317 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.414282 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/bond-cni-plugin/0.log" Jul 23 21:12:09.438975 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.438946 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/routeoverride-cni/0.log" Jul 23 21:12:09.461973 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.461937 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/whereabouts-cni-bincopy/0.log" Jul 23 21:12:09.487759 ip-10-0-129-43 kubenswrapper[2578]: I0723 21:12:09.487733 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2cfh_549e7963-6203-406f-9c29-2716303002d5/whereabouts-cni/0.log"