Jul 07 19:19:05.388937 ip-10-0-135-242 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 19:19:05.388952 ip-10-0-135-242 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 19:19:05.388961 ip-10-0-135-242 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 19:19:05.389281 ip-10-0-135-242 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 19:19:15.482454 ip-10-0-135-242 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 19:19:15.482470 ip-10-0-135-242 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2d2ce824e97942a8b5d30a65848a76ab -- Jul 07 19:21:24.076429 ip-10-0-135-242 systemd[1]: Starting Kubernetes Kubelet... Jul 07 19:21:24.565418 ip-10-0-135-242 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 19:21:24.565418 ip-10-0-135-242 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 19:21:24.565418 ip-10-0-135-242 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 19:21:24.565418 ip-10-0-135-242 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 19:21:24.565418 ip-10-0-135-242 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 19:21:24.568314 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.568212 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 19:21:24.574370 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574347 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:21:24.574370 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574368 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:21:24.574370 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574372 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574376 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574379 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574383 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574386 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574389 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574392 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574395 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574399 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574401 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574404 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574407 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574410 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574413 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574416 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574418 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574421 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574424 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574427 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:21:24.574479 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574429 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574433 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574441 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574444 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574446 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574449 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574452 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574455 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574458 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574461 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574464 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574467 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574470 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574473 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574476 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574481 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574486 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574491 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574495 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574498 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:21:24.574964 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574501 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574504 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574506 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574509 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574513 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574516 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574518 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574521 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574524 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574527 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574529 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574532 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574537 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574540 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574544 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574547 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574550 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574553 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574557 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:21:24.575517 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574560 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574563 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574566 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574569 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574572 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574575 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574577 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574580 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574583 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574587 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574589 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574593 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574596 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574598 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574601 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574604 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574607 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574609 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574612 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574614 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:21:24.576010 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574617 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574620 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574622 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574625 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574628 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574630 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574633 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574637 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574640 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574643 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574647 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574649 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574654 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574656 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574659 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574662 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574665 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574667 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:21:24.576547 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.574671 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:21:24.577729 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577714 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:21:24.577729 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577728 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577732 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577735 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577738 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577742 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577745 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577748 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577750 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577756 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577760 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577764 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577767 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577770 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577773 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577776 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577779 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577783 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577786 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577789 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577792 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:21:24.577798 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577795 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577798 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577801 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577805 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577809 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577813 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577816 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577819 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577822 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577827 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577830 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577833 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577836 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577839 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577842 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577845 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577848 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577852 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577855 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:21:24.578386 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577858 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577860 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577863 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577865 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577868 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577871 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577873 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577876 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577878 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577881 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577883 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577886 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577889 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577891 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577896 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577901 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577904 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577907 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577910 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:21:24.578875 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577913 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577916 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577919 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577922 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577925 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577928 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577930 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577933 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577936 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577938 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577941 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577944 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577948 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577951 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577954 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577956 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577959 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577962 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577965 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577967 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:21:24.579391 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577970 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577973 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577976 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577979 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577981 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577984 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577986 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577990 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577994 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577997 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.577999 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578002 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578005 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578008 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578011 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578014 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578016 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578020 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578022 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578025 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:21:24.579923 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578473 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578479 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578482 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578487 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578492 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578495 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578498 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578501 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578504 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578506 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578509 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578512 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578515 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578517 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578520 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578523 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578526 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578529 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578531 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578535 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:21:24.580471 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578539 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578542 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578545 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578548 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578551 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578554 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578557 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578560 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578562 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578565 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578567 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578570 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578573 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578576 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578579 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578581 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578584 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578588 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578590 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578593 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:21:24.580987 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578596 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578599 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578602 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578605 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578608 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578610 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578613 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578615 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578618 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578621 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578624 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578628 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578630 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578633 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578636 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578638 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578641 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578644 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578647 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:21:24.581537 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578649 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578652 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578655 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578657 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578662 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578666 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578669 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578672 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578675 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578678 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578681 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578684 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578687 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578690 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578692 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578695 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578698 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578700 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578703 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578707 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:21:24.582031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578710 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578713 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578716 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578719 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578722 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578725 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578728 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578731 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578734 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578737 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578739 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578742 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578745 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578747 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578750 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578753 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578755 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578758 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578761 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.578765 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:21:24.582562 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578861 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578873 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578884 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578892 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578904 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578910 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578915 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578920 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578924 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578927 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578931 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578935 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578938 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578942 2580 flags.go:64] FLAG: --cgroup-root="" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578945 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578949 2580 flags.go:64] FLAG: --client-ca-file="" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578952 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578955 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578960 2580 flags.go:64] FLAG: --cluster-domain="" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578963 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578966 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578970 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578974 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578978 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 19:21:24.583088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578981 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578984 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578988 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578991 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578995 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.578998 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579001 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579005 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579010 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579013 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579017 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579020 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579023 2580 flags.go:64] FLAG: --enable-server="true" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579027 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579033 2580 flags.go:64] FLAG: --event-burst="100" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579036 2580 flags.go:64] FLAG: --event-qps="50" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579039 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579061 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579064 2580 flags.go:64] FLAG: --eviction-hard="" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579069 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579072 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579075 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579079 2580 flags.go:64] FLAG: --eviction-soft="" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579082 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579086 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 19:21:24.583760 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579089 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579092 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579095 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579098 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579101 2580 flags.go:64] FLAG: --feature-gates="" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579105 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579109 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579112 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579115 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579119 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579122 2580 flags.go:64] FLAG: --help="false" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579125 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-135-242.ec2.internal" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579129 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579133 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579136 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579139 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579143 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579146 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579150 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579153 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579156 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579160 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579163 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579167 2580 flags.go:64] FLAG: --kube-reserved="" Jul 07 19:21:24.584471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579173 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579176 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579180 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579183 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579187 2580 flags.go:64] FLAG: --lock-file="" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579190 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579193 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579197 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579203 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579206 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579210 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579213 2580 flags.go:64] FLAG: --logging-format="text" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579216 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579219 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579223 2580 flags.go:64] FLAG: --manifest-url="" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579226 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579231 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579234 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579239 2580 flags.go:64] FLAG: --max-pods="110" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579242 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579245 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579248 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579252 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579255 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579258 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 19:21:24.585116 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579261 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579269 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579273 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579276 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579279 2580 flags.go:64] FLAG: --pod-cidr="" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579283 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579290 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579293 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579298 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579301 2580 flags.go:64] FLAG: --port="10250" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579305 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579310 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03379a45ab3b7e841" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579313 2580 flags.go:64] FLAG: --qos-reserved="" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579316 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579319 2580 flags.go:64] FLAG: --register-node="true" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579322 2580 flags.go:64] FLAG: --register-with-taints="" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579326 2580 flags.go:64] FLAG: --registry-burst="10" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579330 2580 flags.go:64] FLAG: --registry-qps="5" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579333 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579336 2580 flags.go:64] FLAG: --reserved-memory="" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579340 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579343 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579346 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579349 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579352 2580 flags.go:64] FLAG: --runonce="false" Jul 07 19:21:24.585766 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579355 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579359 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579362 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579365 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579368 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579371 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579374 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579378 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579381 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579384 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579388 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579391 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579394 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579405 2580 flags.go:64] FLAG: --system-cgroups="" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579408 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579416 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579419 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579422 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579427 2580 flags.go:64] FLAG: --tls-min-version="" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579430 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579433 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579436 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579439 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579443 2580 flags.go:64] FLAG: --v="2" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579447 2580 flags.go:64] FLAG: --version="false" Jul 07 19:21:24.586583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579451 2580 flags.go:64] FLAG: --vmodule="" Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579455 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.579459 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579610 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579615 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579618 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579621 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579624 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579627 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579630 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579633 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579636 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579639 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579642 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579645 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579647 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579650 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579653 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579655 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579658 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:21:24.587305 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579661 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579664 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579669 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579671 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579674 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579678 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579681 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579684 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579687 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579690 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579692 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579695 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579698 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579701 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579704 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579707 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579709 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579712 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579715 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579718 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:21:24.587866 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579720 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579723 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579725 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579728 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579731 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579733 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579736 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579739 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579741 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579744 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579747 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579750 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579753 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579757 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579764 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579767 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579770 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579774 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579777 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:21:24.588418 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579780 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579782 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579785 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579788 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579791 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579793 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579796 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579799 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579802 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579805 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579815 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579820 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579823 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579826 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579829 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579832 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579835 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579838 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579840 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579843 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:21:24.588938 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579846 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579850 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579854 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579857 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579860 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579863 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579866 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579871 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579875 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579878 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579883 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579886 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579890 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579893 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579896 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579899 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579901 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579905 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579908 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:21:24.589482 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579910 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:21:24.589979 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579913 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:21:24.589979 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579916 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:21:24.589979 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.579919 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:21:24.589979 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.580766 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 19:21:24.589979 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.589155 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 19:21:24.589979 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.589176 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 19:21:24.590403 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.590390 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 19:21:24.590439 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.590402 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 19:21:24.590481 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590472 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:21:24.590481 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590480 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590483 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590487 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590492 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590496 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590500 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590503 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590506 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590509 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590512 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590515 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590517 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590520 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590523 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590526 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590529 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590531 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590534 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590537 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:21:24.590535 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590540 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590543 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590546 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590549 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590551 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590555 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590558 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590560 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590563 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590572 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590575 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590578 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590580 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590583 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590587 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590591 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590595 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590597 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590600 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590603 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:21:24.591031 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590605 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590608 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590610 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590613 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590616 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590618 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590621 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590624 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590626 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590629 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590631 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590634 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590637 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590639 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590644 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590647 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590649 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590652 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590655 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590657 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:21:24.591571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590660 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590663 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590671 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590674 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590677 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590679 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590682 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590685 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590687 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590690 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590693 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590695 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590700 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590703 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590706 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590709 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590712 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590715 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590718 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:21:24.592110 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590720 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590723 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590725 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590728 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590731 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590733 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590736 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590739 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590742 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590745 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590748 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590750 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590753 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590756 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590758 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590761 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590771 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590774 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590777 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:21:24.592598 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590779 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.590785 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590954 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590960 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590964 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590967 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590970 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590973 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590976 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590979 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590982 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590985 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590988 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590990 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590993 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:21:24.593113 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590996 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.590999 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591001 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591004 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591007 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591010 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591013 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591016 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591019 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591022 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591024 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591027 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591030 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591033 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591037 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591061 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591065 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591068 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591071 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:21:24.593522 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591074 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591078 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591082 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591085 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591088 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591091 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591094 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591097 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591100 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591103 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591106 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591109 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591111 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591114 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591117 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591119 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591122 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591125 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591128 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:21:24.594186 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591131 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591133 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591136 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591139 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591141 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591144 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591147 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591150 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591152 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591155 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591165 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591168 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591171 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591173 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591176 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591181 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591184 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591187 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591190 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591193 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:21:24.594699 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591195 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591198 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591201 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591204 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591206 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591209 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591212 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591215 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591217 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591220 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591223 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591227 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591230 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591232 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591235 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591237 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591240 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591243 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591245 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591248 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:21:24.595231 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591250 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591253 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591262 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591271 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591274 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591276 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591279 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:24.591281 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.591286 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.591439 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.593865 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 19:21:24.595742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.595671 2580 server.go:1019] "Starting client certificate rotation" Jul 07 19:21:24.596038 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.595790 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 19:21:24.596038 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.595840 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 19:21:24.598772 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.598756 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 19:21:24.609311 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.609283 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 19:21:24.621742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.621710 2580 log.go:25] "Validated CRI v1 runtime API" Jul 07 19:21:24.624181 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.624163 2580 log.go:25] "Validated CRI v1 image API" Jul 07 19:21:24.626109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.626091 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 19:21:24.627539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.627519 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 19:21:24.629080 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.629060 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d0589342-d49b-4e20-9bad-f0bb4f247f4e:/dev/nvme0n1p3 e2f9cc84-9e3c-40e6-8e3a-9f52e3f302dd:/dev/nvme0n1p4] Jul 07 19:21:24.629140 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.629080 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 19:21:24.635090 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.634948 2580 manager.go:216] Machine: {Timestamp:2026-07-07 19:21:24.633253891 +0000 UTC m=+0.418238886 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3132926 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b6d9f7634c808ed69ebba99e0e65c SystemUUID:ec2b6d9f-7634-c808-ed69-ebba99e0e65c BootID:2d2ce824-e979-42a8-b5d3-0a65848a76ab Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:15:32:27:a9:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:15:32:27:a9:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:5a:87:64:11:97 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 19:21:24.635090 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.635087 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 19:21:24.635217 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.635186 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 19:21:24.639440 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.639407 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 19:21:24.639595 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.639443 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-242.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 19:21:24.640803 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.640790 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 19:21:24.640841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.640805 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 19:21:24.640841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.640813 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 19:21:24.640841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.640829 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 19:21:24.643028 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.643014 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 19:21:24.646654 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646639 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 07 19:21:24.646813 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646803 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 19:21:24.646851 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646836 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 07 19:21:24.646885 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646855 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 19:21:24.646885 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646876 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 19:21:24.646942 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646887 2580 kubelet.go:398] "Adding apiserver pod source" Jul 07 19:21:24.646942 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.646897 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 19:21:24.647478 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.647447 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 19:21:24.647478 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.647476 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 19:21:24.650310 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.650282 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 19:21:24.652796 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.652779 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 19:21:24.652844 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.652814 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 19:21:24.654244 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654232 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 19:21:24.654299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654251 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 19:21:24.654299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654257 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 19:21:24.654299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654266 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 19:21:24.654299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654280 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 19:21:24.654299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654289 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 19:21:24.654299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654298 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 19:21:24.654463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654305 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 19:21:24.654463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654313 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 19:21:24.654463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654319 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 19:21:24.654463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654329 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 19:21:24.654463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.654342 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 19:21:24.655267 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.655255 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 19:21:24.655267 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.655267 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 19:21:24.658567 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.658544 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-swb9v" Jul 07 19:21:24.658861 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.658846 2580 server.go:1268] "Started kubelet" Jul 07 19:21:24.659017 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.658960 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 19:21:24.659280 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.658943 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 19:21:24.659371 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.659323 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 19:21:24.659890 ip-10-0-135-242 systemd[1]: Started Kubernetes Kubelet. Jul 07 19:21:24.662423 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.661826 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 19:21:24.662423 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.662241 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 19:21:24.662423 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.662392 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 19:21:24.662605 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.662589 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-242.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 19:21:24.665015 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.664996 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 07 19:21:24.665409 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.665371 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-swb9v" Jul 07 19:21:24.665729 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.665699 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 19:21:24.668153 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.667006 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-242.ec2.internal.18c01838cc04b35f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-242.ec2.internal,UID:ip-10-0-135-242.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-242.ec2.internal,},FirstTimestamp:2026-07-07 19:21:24.658811743 +0000 UTC m=+0.443796736,LastTimestamp:2026-07-07 19:21:24.658811743 +0000 UTC m=+0.443796736,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-242.ec2.internal,}" Jul 07 19:21:24.672560 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.672543 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 19:21:24.672773 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.672659 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 19:21:24.672975 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.672947 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:24.673511 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.673489 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 19:21:24.673770 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.673750 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 19:21:24.673857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.673775 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 19:21:24.673909 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.673891 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 07 19:21:24.673909 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.673901 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 07 19:21:24.674218 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674196 2580 factory.go:55] Registering systemd factory Jul 07 19:21:24.674218 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674220 2580 factory.go:223] Registration of the systemd container factory successfully Jul 07 19:21:24.674500 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674482 2580 factory.go:153] Registering CRI-O factory Jul 07 19:21:24.674500 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674505 2580 factory.go:223] Registration of the crio container factory successfully Jul 07 19:21:24.674624 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674561 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 19:21:24.674624 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674592 2580 factory.go:103] Registering Raw factory Jul 07 19:21:24.674624 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.674607 2580 manager.go:1196] Started watching for new ooms in manager Jul 07 19:21:24.675298 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.675280 2580 manager.go:349] Starting recovery of all containers Jul 07 19:21:24.678121 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.678098 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:24.685695 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.685481 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-242.ec2.internal\" not found" node="ip-10-0-135-242.ec2.internal" Jul 07 19:21:24.686890 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.686874 2580 manager.go:354] Recovery completed Jul 07 19:21:24.692092 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.692077 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:24.694232 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694209 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:24.694304 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694251 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:24.694304 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694267 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:24.694822 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694808 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 19:21:24.694822 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694819 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 19:21:24.694927 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694837 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 07 19:21:24.694927 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.694849 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:24.696278 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.696266 2580 policy_none.go:49] "None policy: Start" Jul 07 19:21:24.696318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.696282 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 19:21:24.696318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.696292 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 19:21:24.696924 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.696909 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:24.697004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.696939 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:24.697004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.696951 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:24.697554 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.697542 2580 policy_none.go:47] "Start" Jul 07 19:21:24.748841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.748799 2580 manager.go:337] "Starting Device Plugin manager" Jul 07 19:21:24.749010 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.748860 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 19:21:24.749010 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.748873 2580 server.go:87] "Starting device plugin registration server" Jul 07 19:21:24.749246 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.749232 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 19:21:24.749346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.749250 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 19:21:24.749744 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.749498 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 19:21:24.749744 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.749597 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 19:21:24.749744 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.749607 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 19:21:24.750059 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.750041 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 19:21:24.750131 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.750092 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:24.851312 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.851200 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:24.853143 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.852466 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:24.853143 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.852506 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:24.853143 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.852521 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:24.853143 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.852549 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-242.ec2.internal" Jul 07 19:21:24.862100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.862065 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-242.ec2.internal" Jul 07 19:21:24.862231 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.862108 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-242.ec2.internal\": node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:24.877599 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.877568 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:24.908431 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.908380 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 19:21:24.909762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.909741 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 19:21:24.909838 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.909786 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 19:21:24.909838 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.909820 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 19:21:24.909917 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.909855 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 19:21:24.912807 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:24.912781 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:24.978390 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:24.978358 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:25.012088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.012048 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal"] Jul 07 19:21:25.012199 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.012156 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:25.014161 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.014139 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:25.014269 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.014171 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:25.014269 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.014182 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:25.015712 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.015699 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:25.015840 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.015826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.015889 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.015858 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:25.016669 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.016650 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:25.016762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.016654 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:25.016762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.016707 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:25.016762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.016720 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:25.016762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.016681 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:25.016762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.016757 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:25.018412 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.018391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.018524 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.018419 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:21:25.019237 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.019222 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:21:25.019343 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.019248 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:21:25.019343 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.019259 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:21:25.034796 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.034765 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-242.ec2.internal\" not found" node="ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.039183 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.039165 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-242.ec2.internal\" not found" node="ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.077140 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.077104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b89d234948b15c1959046b4ec18ba250-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.ec2.internal\" (UID: \"b89d234948b15c1959046b4ec18ba250\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.077267 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.077146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ae2a87f2e0e6ded57445e1e57e2d617-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal\" (UID: \"4ae2a87f2e0e6ded57445e1e57e2d617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.077267 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.077169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ae2a87f2e0e6ded57445e1e57e2d617-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal\" (UID: \"4ae2a87f2e0e6ded57445e1e57e2d617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.079356 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.079335 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:25.177400 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.177332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ae2a87f2e0e6ded57445e1e57e2d617-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal\" (UID: \"4ae2a87f2e0e6ded57445e1e57e2d617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.177400 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.177384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ae2a87f2e0e6ded57445e1e57e2d617-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal\" (UID: \"4ae2a87f2e0e6ded57445e1e57e2d617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.177570 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.177404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b89d234948b15c1959046b4ec18ba250-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.ec2.internal\" (UID: \"b89d234948b15c1959046b4ec18ba250\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.177570 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.177453 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ae2a87f2e0e6ded57445e1e57e2d617-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal\" (UID: \"4ae2a87f2e0e6ded57445e1e57e2d617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.177570 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.177484 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b89d234948b15c1959046b4ec18ba250-config\") pod \"kube-apiserver-proxy-ip-10-0-135-242.ec2.internal\" (UID: \"b89d234948b15c1959046b4ec18ba250\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.177570 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.177456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ae2a87f2e0e6ded57445e1e57e2d617-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal\" (UID: \"4ae2a87f2e0e6ded57445e1e57e2d617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.179561 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.179539 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:25.280477 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.280437 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:25.342407 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.342364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.344715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.344697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.381465 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.381429 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-242.ec2.internal\" not found" Jul 07 19:21:25.477836 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.477750 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:25.576505 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.576456 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.591883 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.591857 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 19:21:25.593786 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.593770 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.595237 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.595220 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 19:21:25.595375 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.595355 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 19:21:25.595422 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.595401 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 19:21:25.595454 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.595404 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 19:21:25.595454 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.595437 2580 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://aa3fb6cc0e53b47a39f4ebe88e1dddaa-81b6ebb866844294.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.135.242:34344->100.59.133.170:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:25.650796 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.650752 2580 apiserver.go:52] "Watching apiserver" Jul 07 19:21:25.665309 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.665279 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 19:21:25.665675 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.665652 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gs7hm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal","openshift-multus/multus-additional-cni-plugins-6gn64","openshift-network-operator/iptables-alerter-wz6q5","openshift-ovn-kubernetes/ovnkube-node-dmrlr","kube-system/konnectivity-agent-j8cqf","openshift-cluster-node-tuning-operator/tuned-sthxf","openshift-dns/node-resolver-vjbsv","openshift-image-registry/node-ca-zgvz5","openshift-multus/multus-9lg42","openshift-multus/network-metrics-daemon-wwmc8","openshift-network-diagnostics/network-check-target-lks2m"] Jul 07 19:21:25.667508 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.667491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.667590 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.667573 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:25.668997 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.668966 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.670551 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.670532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.671700 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.671674 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 19:21:25.671998 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.671977 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.672108 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.671997 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.672108 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.672085 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k5shc\"" Jul 07 19:21:25.672966 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.672946 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 19:21:25.673930 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.673880 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 19:16:24 +0000 UTC" deadline="2027-12-13 18:06:31.196542137 +0000 UTC" Jul 07 19:21:25.674017 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.673938 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12574h45m5.522614513s" Jul 07 19:21:25.674105 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674025 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 19:21:25.674105 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674036 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.674274 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674249 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 19:21:25.674385 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674369 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.674488 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674434 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 19:21:25.674607 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.674688 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.674564 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2bdjg\"" Jul 07 19:21:25.677140 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.677125 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.678344 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.678323 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.678344 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.678345 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.678587 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.678570 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 19:21:25.678650 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.678577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.678705 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.678684 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4rtlw\"" Jul 07 19:21:25.680293 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.680275 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.680881 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.680865 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 19:21:25.681146 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.681130 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 19:21:25.681234 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.681132 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 19:21:25.681234 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.681199 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zmxrp\"" Jul 07 19:21:25.681663 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.681648 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.682169 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.682152 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.683187 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683172 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.683257 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cnibin\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683312 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-os-release\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683312 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2zh7\" (UniqueName: \"kubernetes.io/projected/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-kube-api-access-p2zh7\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683412 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45d31af1-5ec3-41c2-890c-41da611bf652-host-slash\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.683412 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.683412 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683528 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683440 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-device-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683528 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683528 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqldx\" (UniqueName: \"kubernetes.io/projected/45d31af1-5ec3-41c2-890c-41da611bf652-kube-api-access-wqldx\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-registration-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be412fb-041b-44dc-b9a9-fbd8792bfd19-dbus\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-sys-fs\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k59dz\" (UniqueName: \"kubernetes.io/projected/7225d0c5-128a-45ac-86de-9c327600e67e-kube-api-access-k59dz\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683905 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683672 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45d31af1-5ec3-41c2-890c-41da611bf652-iptables-alerter-script\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.683905 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be412fb-041b-44dc-b9a9-fbd8792bfd19-kubelet-config\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.683905 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-socket-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683905 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683721 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-etc-selinux\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.683905 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.683755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-system-cni-dir\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.684456 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.684437 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.684608 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.684593 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.684900 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.684793 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k6kcl\"" Jul 07 19:21:25.684900 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.684793 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 19:21:25.684900 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.684845 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.685145 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.684994 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.685933 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.685915 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 19:21:25.685933 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.685926 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.685929 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.685930 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.686004 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g7dp9\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.686057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9j6bp\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.685916 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.686101 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 19:21:25.686176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.685935 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 19:21:25.686575 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.686477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gvw8z\"" Jul 07 19:21:25.686575 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.686512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:25.686667 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.686588 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:25.687064 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.687049 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rb68r\"" Jul 07 19:21:25.688151 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.688131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:25.688231 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.688197 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:25.688668 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.688649 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 19:21:25.690402 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.690383 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 19:21:25.774708 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.774685 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 19:21:25.783106 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.783079 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4wsd" Jul 07 19:21:25.783964 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.783943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.784052 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.783974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.784052 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqn8m\" (UniqueName: \"kubernetes.io/projected/9384fe94-f3cc-41ae-a897-53bc166f00b6-kube-api-access-vqn8m\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:25.784052 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.784182 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-multus-certs\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.784182 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784093 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be412fb-041b-44dc-b9a9-fbd8792bfd19-dbus\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.784182 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.784140 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:25.784182 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k59dz\" (UniqueName: \"kubernetes.io/projected/7225d0c5-128a-45ac-86de-9c327600e67e-kube-api-access-k59dz\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.784182 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784166 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be412fb-041b-44dc-b9a9-fbd8792bfd19-dbus\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.784420 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4phbc\" (UniqueName: \"kubernetes.io/projected/574fea1d-1d1e-4237-a70d-47f8350db209-kube-api-access-4phbc\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.784420 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.784367 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:26.28429407 +0000 UTC m=+2.069279071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:25.784420 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:25.784588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-cni-bin\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.784588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-k8s-cni-cncf-io\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.784588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-conf-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.784588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-etc-selinux\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.784588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cnibin\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574fea1d-1d1e-4237-a70d-47f8350db209-tmp-dir\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-ovn\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784630 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cnibin\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784644 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-etc-selinux\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-run-ovn-kubernetes\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6hnz\" (UniqueName: \"kubernetes.io/projected/6dd393af-8584-428f-99b8-2e4fa5f6a86f-kube-api-access-d6hnz\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hgv4\" (UniqueName: \"kubernetes.io/projected/7cbfa312-3399-4119-9e04-accaaf744ab6-kube-api-access-6hgv4\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784768 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-cni-binary-copy\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.784791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-slash\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-log-socket\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784836 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-cni-netd\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdwgk\" (UniqueName: \"kubernetes.io/projected/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-kube-api-access-jdwgk\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-registration-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-run-netns\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-registration-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.784974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-cnibin\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45d31af1-5ec3-41c2-890c-41da611bf652-iptables-alerter-script\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovnkube-config\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e20f31af-c455-4ddf-92ad-db1374f312e1-agent-certs\") pod \"konnectivity-agent-j8cqf\" (UID: \"e20f31af-c455-4ddf-92ad-db1374f312e1\") " pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-systemd\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-socket-dir-parent\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-daemon-config\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-modprobe-d\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-system-cni-dir\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-os-release\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785301 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-system-cni-dir\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45d31af1-5ec3-41c2-890c-41da611bf652-host-slash\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqldx\" (UniqueName: \"kubernetes.io/projected/45d31af1-5ec3-41c2-890c-41da611bf652-kube-api-access-wqldx\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-env-overrides\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785371 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-os-release\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovn-node-metrics-cert\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e20f31af-c455-4ddf-92ad-db1374f312e1-konnectivity-ca\") pod \"konnectivity-agent-j8cqf\" (UID: \"e20f31af-c455-4ddf-92ad-db1374f312e1\") " pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45d31af1-5ec3-41c2-890c-41da611bf652-host-slash\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-device-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.785695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785540 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-device-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-kubelet\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-etc-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysctl-conf\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785634 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45d31af1-5ec3-41c2-890c-41da611bf652-iptables-alerter-script\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785620 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-host\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-tuned\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-var-lib-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785742 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-run\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54dd69a2-f5ce-4931-8a98-fd2065361bed-tmp\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-system-cni-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-cni-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.785948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-cni-bin\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-sys-fs\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:25.786495 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786078 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786087 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-sys-fs\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysconfig\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-kubernetes\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7cbfa312-3399-4119-9e04-accaaf744ab6-host\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7cbfa312-3399-4119-9e04-accaaf744ab6-serviceca\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be412fb-041b-44dc-b9a9-fbd8792bfd19-kubelet-config\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2zh7\" (UniqueName: \"kubernetes.io/projected/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-kube-api-access-p2zh7\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-systemd-units\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-cni-multus\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-lib-modules\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mzjk\" (UniqueName: \"kubernetes.io/projected/54dd69a2-f5ce-4931-8a98-fd2065361bed-kube-api-access-2mzjk\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be412fb-041b-44dc-b9a9-fbd8792bfd19-kubelet-config\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-netns\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-hostroot\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.787264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/574fea1d-1d1e-4237-a70d-47f8350db209-hosts-file\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-node-log\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovnkube-script-lib\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-os-release\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-systemd\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysctl-d\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-etc-kubernetes\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786680 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-socket-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786712 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-sys\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-var-lib-kubelet\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-kubelet\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.787857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.786856 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7225d0c5-128a-45ac-86de-9c327600e67e-socket-dir\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.792640 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.792621 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4wsd" Jul 07 19:21:25.795892 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.795869 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 19:21:25.799344 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.799322 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k59dz\" (UniqueName: \"kubernetes.io/projected/7225d0c5-128a-45ac-86de-9c327600e67e-kube-api-access-k59dz\") pod \"aws-ebs-csi-driver-node-5jfpt\" (UID: \"7225d0c5-128a-45ac-86de-9c327600e67e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:25.800554 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.800529 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqldx\" (UniqueName: \"kubernetes.io/projected/45d31af1-5ec3-41c2-890c-41da611bf652-kube-api-access-wqldx\") pod \"iptables-alerter-wz6q5\" (UID: \"45d31af1-5ec3-41c2-890c-41da611bf652\") " pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:25.803775 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.803756 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2zh7\" (UniqueName: \"kubernetes.io/projected/ceba1e12-77ed-44ff-8d8d-688fba7c2cb8-kube-api-access-p2zh7\") pod \"multus-additional-cni-plugins-6gn64\" (UID: \"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8\") " pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:25.887711 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-cni-netd\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887711 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887710 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdwgk\" (UniqueName: \"kubernetes.io/projected/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-kube-api-access-jdwgk\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-run-netns\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-cnibin\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887793 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-cni-netd\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovnkube-config\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887820 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-run-netns\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887831 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e20f31af-c455-4ddf-92ad-db1374f312e1-agent-certs\") pod \"konnectivity-agent-j8cqf\" (UID: \"e20f31af-c455-4ddf-92ad-db1374f312e1\") " pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-cnibin\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-systemd\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-socket-dir-parent\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-daemon-config\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.887965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.887965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-modprobe-d\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888020 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-systemd\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-env-overrides\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-modprobe-d\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovn-node-metrics-cert\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e20f31af-c455-4ddf-92ad-db1374f312e1-konnectivity-ca\") pod \"konnectivity-agent-j8cqf\" (UID: \"e20f31af-c455-4ddf-92ad-db1374f312e1\") " pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-kubelet\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-etc-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysctl-conf\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-kubelet\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-host\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-tuned\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-var-lib-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-run\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888344 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-etc-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54dd69a2-f5ce-4931-8a98-fd2065361bed-tmp\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-system-cni-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-cni-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.888672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-cni-bin\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysctl-conf\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysconfig\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-kubernetes\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-host\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7cbfa312-3399-4119-9e04-accaaf744ab6-host\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888607 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7cbfa312-3399-4119-9e04-accaaf744ab6-host\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888613 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-daemon-config\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888608 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-cni-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7cbfa312-3399-4119-9e04-accaaf744ab6-serviceca\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-cni-bin\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-systemd-units\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.888714 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888729 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-var-lib-openvswitch\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888770 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-systemd-units\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.888787 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:26.388768716 +0000 UTC m=+2.173753702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888794 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysconfig\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.889659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888060 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-socket-dir-parent\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888823 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-run\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888846 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-kubernetes\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.888729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-cni-multus\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-lib-modules\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mzjk\" (UniqueName: \"kubernetes.io/projected/54dd69a2-f5ce-4931-8a98-fd2065361bed-kube-api-access-2mzjk\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-netns\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-hostroot\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/574fea1d-1d1e-4237-a70d-47f8350db209-hosts-file\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-node-log\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889260 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7cbfa312-3399-4119-9e04-accaaf744ab6-serviceca\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889283 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovnkube-script-lib\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889288 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-lib-modules\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-os-release\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-systemd\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysctl-d\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889357 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-system-cni-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889581 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-node-log\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.890743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889634 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-netns\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889676 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-hostroot\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889726 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/574fea1d-1d1e-4237-a70d-47f8350db209-hosts-file\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-etc-kubernetes\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-sys\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-var-lib-kubelet\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889940 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-systemd\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-kubelet\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqn8m\" (UniqueName: \"kubernetes.io/projected/9384fe94-f3cc-41ae-a897-53bc166f00b6-kube-api-access-vqn8m\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890019 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-sys\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-multus-certs\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-os-release\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4phbc\" (UniqueName: \"kubernetes.io/projected/574fea1d-1d1e-4237-a70d-47f8350db209-kube-api-access-4phbc\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890119 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-cni-bin\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.891771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-k8s-cni-cncf-io\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890167 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-cni-multus\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-conf-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574fea1d-1d1e-4237-a70d-47f8350db209-tmp-dir\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890220 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-multus-certs\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890247 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-sysctl-d\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-ovn\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.889962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e20f31af-c455-4ddf-92ad-db1374f312e1-konnectivity-ca\") pod \"konnectivity-agent-j8cqf\" (UID: \"e20f31af-c455-4ddf-92ad-db1374f312e1\") " pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890266 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-cni-bin\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890183 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovnkube-script-lib\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890086 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-etc-kubernetes\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-run-ovn\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890315 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54dd69a2-f5ce-4931-8a98-fd2065361bed-var-lib-kubelet\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890347 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-multus-conf-dir\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-var-lib-kubelet\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890390 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-host-run-k8s-cni-cncf-io\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-run-ovn-kubernetes\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6hnz\" (UniqueName: \"kubernetes.io/projected/6dd393af-8584-428f-99b8-2e4fa5f6a86f-kube-api-access-d6hnz\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890503 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hgv4\" (UniqueName: \"kubernetes.io/projected/7cbfa312-3399-4119-9e04-accaaf744ab6-kube-api-access-6hgv4\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-cni-binary-copy\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890558 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-run-ovn-kubernetes\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-slash\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-log-socket\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.890686 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-host-slash\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-env-overrides\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891237 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6dd393af-8584-428f-99b8-2e4fa5f6a86f-log-socket\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891304 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovnkube-config\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54dd69a2-f5ce-4931-8a98-fd2065361bed-etc-tuned\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-cni-binary-copy\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891550 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54dd69a2-f5ce-4931-8a98-fd2065361bed-tmp\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891601 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574fea1d-1d1e-4237-a70d-47f8350db209-tmp-dir\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.891632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e20f31af-c455-4ddf-92ad-db1374f312e1-agent-certs\") pod \"konnectivity-agent-j8cqf\" (UID: \"e20f31af-c455-4ddf-92ad-db1374f312e1\") " pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:25.892980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.892299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6dd393af-8584-428f-99b8-2e4fa5f6a86f-ovn-node-metrics-cert\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.900067 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.900035 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdwgk\" (UniqueName: \"kubernetes.io/projected/a40d97de-6f3e-45ca-adbc-c4b5acbce37e-kube-api-access-jdwgk\") pod \"multus-9lg42\" (UID: \"a40d97de-6f3e-45ca-adbc-c4b5acbce37e\") " pod="openshift-multus/multus-9lg42" Jul 07 19:21:25.902587 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.902568 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:25.902687 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.902591 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:25.902687 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.902603 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:25.902808 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:25.902687 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:21:26.402673651 +0000 UTC m=+2.187658631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:25.906164 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.906138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hgv4\" (UniqueName: \"kubernetes.io/projected/7cbfa312-3399-4119-9e04-accaaf744ab6-kube-api-access-6hgv4\") pod \"node-ca-zgvz5\" (UID: \"7cbfa312-3399-4119-9e04-accaaf744ab6\") " pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:25.906342 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.906285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6hnz\" (UniqueName: \"kubernetes.io/projected/6dd393af-8584-428f-99b8-2e4fa5f6a86f-kube-api-access-d6hnz\") pod \"ovnkube-node-dmrlr\" (UID: \"6dd393af-8584-428f-99b8-2e4fa5f6a86f\") " pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:25.906539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.906490 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqn8m\" (UniqueName: \"kubernetes.io/projected/9384fe94-f3cc-41ae-a897-53bc166f00b6-kube-api-access-vqn8m\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:25.906620 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.906604 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mzjk\" (UniqueName: \"kubernetes.io/projected/54dd69a2-f5ce-4931-8a98-fd2065361bed-kube-api-access-2mzjk\") pod \"tuned-sthxf\" (UID: \"54dd69a2-f5ce-4931-8a98-fd2065361bed\") " pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:25.907213 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.907175 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4phbc\" (UniqueName: \"kubernetes.io/projected/574fea1d-1d1e-4237-a70d-47f8350db209-kube-api-access-4phbc\") pod \"node-resolver-vjbsv\" (UID: \"574fea1d-1d1e-4237-a70d-47f8350db209\") " pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:25.974004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:25.973969 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:26.002643 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.002617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" Jul 07 19:21:26.012494 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.012455 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb89d234948b15c1959046b4ec18ba250.slice/crio-5bb15918df04a1d716f853ebc9e9d0dcffde5ce443cc7b0792cd09d2e0b310c3 WatchSource:0}: Error finding container 5bb15918df04a1d716f853ebc9e9d0dcffde5ce443cc7b0792cd09d2e0b310c3: Status 404 returned error can't find the container with id 5bb15918df04a1d716f853ebc9e9d0dcffde5ce443cc7b0792cd09d2e0b310c3 Jul 07 19:21:26.012799 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.012778 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7225d0c5_128a_45ac_86de_9c327600e67e.slice/crio-f9ce96f97f582165fa26db3ebbe7f2cbc04ba3e7dcb489dd8c5a00e5621eeb5a WatchSource:0}: Error finding container f9ce96f97f582165fa26db3ebbe7f2cbc04ba3e7dcb489dd8c5a00e5621eeb5a: Status 404 returned error can't find the container with id f9ce96f97f582165fa26db3ebbe7f2cbc04ba3e7dcb489dd8c5a00e5621eeb5a Jul 07 19:21:26.019321 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.019302 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:21:26.021008 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.020988 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6gn64" Jul 07 19:21:26.028561 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.028540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wz6q5" Jul 07 19:21:26.029132 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.029111 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceba1e12_77ed_44ff_8d8d_688fba7c2cb8.slice/crio-f921d2ff2c7cb00d5fc6a101ce080b3478f9aba2b84f9a17c0844cd8f656033a WatchSource:0}: Error finding container f921d2ff2c7cb00d5fc6a101ce080b3478f9aba2b84f9a17c0844cd8f656033a: Status 404 returned error can't find the container with id f921d2ff2c7cb00d5fc6a101ce080b3478f9aba2b84f9a17c0844cd8f656033a Jul 07 19:21:26.044988 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.044957 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d31af1_5ec3_41c2_890c_41da611bf652.slice/crio-ebea2135816ae8bb90948c8141650c5303085c1ae71e7d8249863b81824e010d WatchSource:0}: Error finding container ebea2135816ae8bb90948c8141650c5303085c1ae71e7d8249863b81824e010d: Status 404 returned error can't find the container with id ebea2135816ae8bb90948c8141650c5303085c1ae71e7d8249863b81824e010d Jul 07 19:21:26.045351 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.045330 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae2a87f2e0e6ded57445e1e57e2d617.slice/crio-a8a736919010902d6558ec5dac5b24181cbc8b569084377d06fc315f54cdb9f0 WatchSource:0}: Error finding container a8a736919010902d6558ec5dac5b24181cbc8b569084377d06fc315f54cdb9f0: Status 404 returned error can't find the container with id a8a736919010902d6558ec5dac5b24181cbc8b569084377d06fc315f54cdb9f0 Jul 07 19:21:26.054011 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.053986 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:26.062932 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.062907 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd393af_8584_428f_99b8_2e4fa5f6a86f.slice/crio-9d5b24af55c272e924818e407a2eae41fb7cc38fa3c29c686487e5e5df4a590a WatchSource:0}: Error finding container 9d5b24af55c272e924818e407a2eae41fb7cc38fa3c29c686487e5e5df4a590a: Status 404 returned error can't find the container with id 9d5b24af55c272e924818e407a2eae41fb7cc38fa3c29c686487e5e5df4a590a Jul 07 19:21:26.071367 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.071334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:26.079287 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.079260 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20f31af_c455_4ddf_92ad_db1374f312e1.slice/crio-3d62519089b6a1642086b4dc2851b362a338395d1ae41b069f433675855610f6 WatchSource:0}: Error finding container 3d62519089b6a1642086b4dc2851b362a338395d1ae41b069f433675855610f6: Status 404 returned error can't find the container with id 3d62519089b6a1642086b4dc2851b362a338395d1ae41b069f433675855610f6 Jul 07 19:21:26.085791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.085771 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sthxf" Jul 07 19:21:26.093695 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.093670 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54dd69a2_f5ce_4931_8a98_fd2065361bed.slice/crio-a59133a0bf4a02d4858781abc141c6559504ddac873b769ec3c46893cc4a45f6 WatchSource:0}: Error finding container a59133a0bf4a02d4858781abc141c6559504ddac873b769ec3c46893cc4a45f6: Status 404 returned error can't find the container with id a59133a0bf4a02d4858781abc141c6559504ddac873b769ec3c46893cc4a45f6 Jul 07 19:21:26.096970 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.096955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vjbsv" Jul 07 19:21:26.105187 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.105165 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574fea1d_1d1e_4237_a70d_47f8350db209.slice/crio-1fa9eec619ed41df5c2cacb0b1eb045636787b4336549c726099db94f4d2fe8b WatchSource:0}: Error finding container 1fa9eec619ed41df5c2cacb0b1eb045636787b4336549c726099db94f4d2fe8b: Status 404 returned error can't find the container with id 1fa9eec619ed41df5c2cacb0b1eb045636787b4336549c726099db94f4d2fe8b Jul 07 19:21:26.124805 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.124775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zgvz5" Jul 07 19:21:26.129324 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.129302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9lg42" Jul 07 19:21:26.135422 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.135388 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbfa312_3399_4119_9e04_accaaf744ab6.slice/crio-ad79935c9dfe9649ebbc3436a7c829eddbf2c1282bf11ce5bb4cc7d692787e51 WatchSource:0}: Error finding container ad79935c9dfe9649ebbc3436a7c829eddbf2c1282bf11ce5bb4cc7d692787e51: Status 404 returned error can't find the container with id ad79935c9dfe9649ebbc3436a7c829eddbf2c1282bf11ce5bb4cc7d692787e51 Jul 07 19:21:26.138375 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:21:26.138353 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40d97de_6f3e_45ca_adbc_c4b5acbce37e.slice/crio-801ded3972bdd5809f00c19f326542948eaf2beea420901494ab91b81bcbe8a5 WatchSource:0}: Error finding container 801ded3972bdd5809f00c19f326542948eaf2beea420901494ab91b81bcbe8a5: Status 404 returned error can't find the container with id 801ded3972bdd5809f00c19f326542948eaf2beea420901494ab91b81bcbe8a5 Jul 07 19:21:26.140288 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.140268 2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-9lg42_openshift-multus(a40d97de-6f3e-45ca-adbc-c4b5acbce37e): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 07 19:21:26.140353 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.140306 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-9lg42" podUID="a40d97de-6f3e-45ca-adbc-c4b5acbce37e" Jul 07 19:21:26.295036 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.294896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:26.295190 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.295090 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:26.295190 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.295165 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:27.295148637 +0000 UTC m=+3.080133616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:26.395689 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.395651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:26.395867 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.395785 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:26.395867 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.395848 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:27.395832088 +0000 UTC m=+3.180817075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:26.497052 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.497016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:26.497250 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.497230 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:26.497307 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.497257 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:26.497307 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.497272 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:26.497392 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.497345 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:21:27.497324045 +0000 UTC m=+3.282309051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:26.796485 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.796413 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:16:25 +0000 UTC" deadline="2028-03-10 10:02:30.721495142 +0000 UTC" Jul 07 19:21:26.796485 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.796486 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14678h41m3.925013203s" Jul 07 19:21:26.926042 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.925957 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" event={"ID":"7225d0c5-128a-45ac-86de-9c327600e67e","Type":"ContainerStarted","Data":"f9ce96f97f582165fa26db3ebbe7f2cbc04ba3e7dcb489dd8c5a00e5621eeb5a"} Jul 07 19:21:26.930959 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.930922 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" event={"ID":"b89d234948b15c1959046b4ec18ba250","Type":"ContainerStarted","Data":"5bb15918df04a1d716f853ebc9e9d0dcffde5ce443cc7b0792cd09d2e0b310c3"} Jul 07 19:21:26.935517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.935478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9lg42" event={"ID":"a40d97de-6f3e-45ca-adbc-c4b5acbce37e","Type":"ContainerStarted","Data":"801ded3972bdd5809f00c19f326542948eaf2beea420901494ab91b81bcbe8a5"} Jul 07 19:21:26.937796 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:26.937758 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-9lg42" podUID="a40d97de-6f3e-45ca-adbc-c4b5acbce37e" Jul 07 19:21:26.941019 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.940976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j8cqf" event={"ID":"e20f31af-c455-4ddf-92ad-db1374f312e1","Type":"ContainerStarted","Data":"3d62519089b6a1642086b4dc2851b362a338395d1ae41b069f433675855610f6"} Jul 07 19:21:26.943178 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.943150 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wz6q5" event={"ID":"45d31af1-5ec3-41c2-890c-41da611bf652","Type":"ContainerStarted","Data":"ebea2135816ae8bb90948c8141650c5303085c1ae71e7d8249863b81824e010d"} Jul 07 19:21:26.947379 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.947349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zgvz5" event={"ID":"7cbfa312-3399-4119-9e04-accaaf744ab6","Type":"ContainerStarted","Data":"ad79935c9dfe9649ebbc3436a7c829eddbf2c1282bf11ce5bb4cc7d692787e51"} Jul 07 19:21:26.962400 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.962357 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vjbsv" event={"ID":"574fea1d-1d1e-4237-a70d-47f8350db209","Type":"ContainerStarted","Data":"1fa9eec619ed41df5c2cacb0b1eb045636787b4336549c726099db94f4d2fe8b"} Jul 07 19:21:26.967003 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.966965 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sthxf" event={"ID":"54dd69a2-f5ce-4931-8a98-fd2065361bed","Type":"ContainerStarted","Data":"a59133a0bf4a02d4858781abc141c6559504ddac873b769ec3c46893cc4a45f6"} Jul 07 19:21:26.984354 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.984313 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"9d5b24af55c272e924818e407a2eae41fb7cc38fa3c29c686487e5e5df4a590a"} Jul 07 19:21:26.991374 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:26.991340 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:27.003059 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.003002 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" event={"ID":"4ae2a87f2e0e6ded57445e1e57e2d617","Type":"ContainerStarted","Data":"a8a736919010902d6558ec5dac5b24181cbc8b569084377d06fc315f54cdb9f0"} Jul 07 19:21:27.023555 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.023512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerStarted","Data":"f921d2ff2c7cb00d5fc6a101ce080b3478f9aba2b84f9a17c0844cd8f656033a"} Jul 07 19:21:27.139310 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.139016 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:27.305394 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.304816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:27.305394 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.304950 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:27.305394 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.305032 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:29.305013844 +0000 UTC m=+5.089998834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:27.405652 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.405561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:27.405822 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.405722 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:27.405822 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.405791 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:29.405771741 +0000 UTC m=+5.190756723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:27.507006 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.506379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:27.507006 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.506543 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:27.507006 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.506562 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:27.507006 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.506574 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:27.507006 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.506642 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:21:29.506621598 +0000 UTC m=+5.291606595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:27.797051 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.796921 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:16:25 +0000 UTC" deadline="2027-12-25 21:29:39.672802914 +0000 UTC" Jul 07 19:21:27.797051 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.796961 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12866h8m11.875846324s" Jul 07 19:21:27.911772 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.910937 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:27.911772 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.911074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:27.911772 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.911466 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:27.911772 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.911556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:27.911772 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:27.911622 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:27.911772 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:27.911724 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:28.436518 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:28.436483 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:21:29.323446 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:29.323263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:29.323937 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.323481 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:29.323937 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.323553 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.323533854 +0000 UTC m=+9.108518841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:29.424292 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:29.424248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:29.424479 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.424400 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:29.424479 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.424476 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.424454213 +0000 UTC m=+9.209439191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:29.525070 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:29.525008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:29.525250 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.525149 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:29.525250 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.525175 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:29.525250 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.525187 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:29.525250 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.525240 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.525224628 +0000 UTC m=+9.310209613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:29.911366 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:29.910783 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:29.911366 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.910941 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:29.911366 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:29.911213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:29.911366 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:29.911216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:29.911366 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.911346 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:29.911777 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:29.911424 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:31.910887 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:31.910856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:31.911329 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:31.910904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:31.911329 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:31.911003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:31.911329 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:31.911059 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:31.911329 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:31.911093 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:31.911329 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:31.911174 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:33.355295 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:33.355256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:33.355772 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.355407 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:33.355772 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.355481 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:41.355466876 +0000 UTC m=+17.140451854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:33.456331 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:33.456295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:33.456505 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.456458 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:33.456555 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.456541 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:41.456519193 +0000 UTC m=+17.241504183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:33.557757 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:33.557718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:33.557936 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.557899 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:33.557936 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.557916 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:33.557936 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.557925 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:33.558104 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.557998 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:21:41.557964566 +0000 UTC m=+17.342949566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:33.910608 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:33.910559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:33.912293 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:33.910903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:33.912293 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.912054 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:33.912293 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.912205 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:33.912293 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:33.912282 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:33.912573 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:33.912383 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:35.910306 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:35.910267 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:35.910763 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:35.910315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:35.910763 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:35.910398 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:35.910763 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:35.910267 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:35.910763 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:35.910520 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:35.910763 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:35.910606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:37.048722 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.048453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sthxf" event={"ID":"54dd69a2-f5ce-4931-8a98-fd2065361bed","Type":"ContainerStarted","Data":"3f1e95fa88f8bcc7c17a3c6c4389c293c8cc022c46f3c23d97b36f1b249ce7b7"} Jul 07 19:21:37.051134 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.051105 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" event={"ID":"b89d234948b15c1959046b4ec18ba250","Type":"ContainerStarted","Data":"334e082909b5b17612e19222aeff85d514618451eecf8e504bc1b5f8911121a1"} Jul 07 19:21:37.051346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.051329 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:37.059737 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.059714 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 19:21:37.060513 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.060493 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal"] Jul 07 19:21:37.085260 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.085205 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" podStartSLOduration=0.085189967 podStartE2EDuration="85.189967ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:37.084879745 +0000 UTC m=+12.869864769" watchObservedRunningTime="2026-07-07 19:21:37.085189967 +0000 UTC m=+12.870174967" Jul 07 19:21:37.085420 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.085325 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sthxf" podStartSLOduration=1.664703458 podStartE2EDuration="12.085315931s" totalImagesPullingTime="10.420612473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:37.071058594 +0000 UTC m=+12.856043595" watchObservedRunningTime="2026-07-07 19:21:37.085315931 +0000 UTC m=+12.870300924" Jul 07 19:21:37.911081 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.911046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:37.911081 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.911069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:37.911335 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:37.911100 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:37.911335 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:37.911174 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:37.911335 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:37.911251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:37.911335 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:37.911264 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:38.055152 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.055118 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zgvz5" event={"ID":"7cbfa312-3399-4119-9e04-accaaf744ab6","Type":"ContainerStarted","Data":"41cdfda1fc5cbcafa8d120c3e528d8e7e2d396977e977c7327e71b3320dc2835"} Jul 07 19:21:38.057056 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.057023 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vjbsv" event={"ID":"574fea1d-1d1e-4237-a70d-47f8350db209","Type":"ContainerStarted","Data":"b8ace6940f9dc306a4409db8bdc4a77f60c1376efa1c19dc01a0b85286f26ce1"} Jul 07 19:21:38.058644 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.058618 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" event={"ID":"4ae2a87f2e0e6ded57445e1e57e2d617","Type":"ContainerStarted","Data":"78ed44b0be122014b35e733bea700be13444ad581c845d708cdcb652cec1a5e9"} Jul 07 19:21:38.060331 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.060310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerStarted","Data":"22ae1af4b2bd4670009cb1f3e3b25f51c13932a754b505e03dbcc406107c6c30"} Jul 07 19:21:38.062022 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.062001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" event={"ID":"7225d0c5-128a-45ac-86de-9c327600e67e","Type":"ContainerStarted","Data":"74dab5559fe144c766a27c4cf4818403d103ec9c3e2266fb7264b07b57ea68c2"} Jul 07 19:21:38.063631 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.063600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j8cqf" event={"ID":"e20f31af-c455-4ddf-92ad-db1374f312e1","Type":"ContainerStarted","Data":"b664bc105895370ce83ffc4cd1185047624255e37dc7394a26dcc600ce57bf17"} Jul 07 19:21:38.063923 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.063903 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:38.073347 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.073290 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zgvz5" podStartSLOduration=2.720657132 podStartE2EDuration="13.073272503s" totalImagesPullingTime="10.352615371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:38.072705611 +0000 UTC m=+13.857690612" watchObservedRunningTime="2026-07-07 19:21:38.073272503 +0000 UTC m=+13.858257503" Jul 07 19:21:38.074270 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.074243 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 19:21:38.074359 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:38.074294 2580 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-242.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-242.ec2.internal" Jul 07 19:21:38.090915 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.090859 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j8cqf" podStartSLOduration=2.673441077 podStartE2EDuration="13.090841417s" totalImagesPullingTime="10.41740034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:38.090418811 +0000 UTC m=+13.875403811" watchObservedRunningTime="2026-07-07 19:21:38.090841417 +0000 UTC m=+13.875826418" Jul 07 19:21:38.107656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:38.107557 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vjbsv" podStartSLOduration=2.71382338 podStartE2EDuration="13.107535954s" totalImagesPullingTime="10.393712574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:38.106862248 +0000 UTC m=+13.891847248" watchObservedRunningTime="2026-07-07 19:21:38.107535954 +0000 UTC m=+13.892520954" Jul 07 19:21:39.067144 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.066867 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wz6q5" event={"ID":"45d31af1-5ec3-41c2-890c-41da611bf652","Type":"ContainerStarted","Data":"0fa684b6fa6edacf3119e2ba1fe677584e892132396b76e4f6997e05e1abd379"} Jul 07 19:21:39.085214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.085169 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wz6q5" podStartSLOduration=3.632889857 podStartE2EDuration="14.085151731s" totalImagesPullingTime="10.452261874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:39.0851353 +0000 UTC m=+14.870120302" watchObservedRunningTime="2026-07-07 19:21:39.085151731 +0000 UTC m=+14.870136745" Jul 07 19:21:39.910914 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.910875 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:39.911102 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.910916 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:39.911102 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.910923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:39.911102 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:39.911028 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:39.911285 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:39.911256 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:39.911351 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:39.911317 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:39.989251 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.989215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:39.989883 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:39.989858 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:40.639068 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:40.639027 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:40.639677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:40.639656 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j8cqf" Jul 07 19:21:41.414636 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:41.414586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:41.414809 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.414731 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:41.414856 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.414819 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:57.414795165 +0000 UTC m=+33.199780161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:41.515644 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:41.515599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:41.515827 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.515722 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:41.515827 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.515796 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:57.515779408 +0000 UTC m=+33.300764388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:41.616162 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:41.616128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:41.616339 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.616297 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:41.616339 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.616323 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:41.616339 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.616335 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:41.616493 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.616403 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:21:57.616384034 +0000 UTC m=+33.401369032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:41.911208 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:41.911071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:41.911208 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:41.911099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:41.911208 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:41.911071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:41.911208 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.911200 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:41.911948 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.911302 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:41.911948 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:41.911382 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:42.075818 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.075574 2580 generic.go:356] "Generic (PLEG): container finished" podID="4ae2a87f2e0e6ded57445e1e57e2d617" containerID="78ed44b0be122014b35e733bea700be13444ad581c845d708cdcb652cec1a5e9" exitCode=0 Jul 07 19:21:42.076109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.075703 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" event={"ID":"4ae2a87f2e0e6ded57445e1e57e2d617","Type":"ContainerDied","Data":"78ed44b0be122014b35e733bea700be13444ad581c845d708cdcb652cec1a5e9"} Jul 07 19:21:42.077832 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.077804 2580 generic.go:356] "Generic (PLEG): container finished" podID="ceba1e12-77ed-44ff-8d8d-688fba7c2cb8" containerID="22ae1af4b2bd4670009cb1f3e3b25f51c13932a754b505e03dbcc406107c6c30" exitCode=0 Jul 07 19:21:42.077944 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.077900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerDied","Data":"22ae1af4b2bd4670009cb1f3e3b25f51c13932a754b505e03dbcc406107c6c30"} Jul 07 19:21:42.080585 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.080544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"305f34d87ee6954a660a7cf685855b4864f38d4e45b87032c9609c369c21690d"} Jul 07 19:21:42.337866 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.337846 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vjbsv_574fea1d-1d1e-4237-a70d-47f8350db209/dns-node-resolver/0.log" Jul 07 19:21:42.355045 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.355025 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 19:21:42.760887 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.760513 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T19:21:42.35504017Z","UUID":"3687c5c4-da20-4209-bb2b-77008730718c","Handler":null,"Name":"","Endpoint":""} Jul 07 19:21:42.762220 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.762190 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 19:21:42.762350 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:42.762233 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 19:21:43.086331 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.086247 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"a6813cdbea3934b8bb82a8ce9148d9eb4a5d6b930e54db25af75b7308022eec2"} Jul 07 19:21:43.086894 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.086347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"0cb26f5922c029a04cf05d93569b57c6c6a21ffadd5326c2bf40569bd9dcbfd9"} Jul 07 19:21:43.086894 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.086402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"4ead29f0cc4e6ac1d101f136fffacea8ade78bbfff872524245f0c547322a08a"} Jul 07 19:21:43.086894 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.086450 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"6051da9118eb5c9c19bcd3f8089b9795ef394548523ad290dca24f697a2f912a"} Jul 07 19:21:43.086894 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.086495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"d8d84bd9afee9dcf2c54fe5e20e5f84ce6f344cb9f614f9cb5cd29d763da7747"} Jul 07 19:21:43.088690 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.088662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" event={"ID":"4ae2a87f2e0e6ded57445e1e57e2d617","Type":"ContainerStarted","Data":"54c335dd7ee56bb280e0deac3d1af883389174e6f2465dc66ea99490ab0c2177"} Jul 07 19:21:43.091078 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.091051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" event={"ID":"7225d0c5-128a-45ac-86de-9c327600e67e","Type":"ContainerStarted","Data":"e7352ff3a90d2eb0f515076ac9976d6942e674af3fd6113d2024ff6ff5bbe392"} Jul 07 19:21:43.109214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.109159 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-242.ec2.internal" podStartSLOduration=18.109138156 podStartE2EDuration="18.109138156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:43.108132162 +0000 UTC m=+18.893117162" watchObservedRunningTime="2026-07-07 19:21:43.109138156 +0000 UTC m=+18.894123159" Jul 07 19:21:43.515015 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.514919 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zgvz5_7cbfa312-3399-4119-9e04-accaaf744ab6/node-ca/0.log" Jul 07 19:21:43.910875 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.910790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:43.910875 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.910816 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:43.911137 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:43.910790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:43.911137 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:43.910926 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:43.911137 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:43.911067 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:43.911242 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:43.911155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:44.095984 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:44.095928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" event={"ID":"7225d0c5-128a-45ac-86de-9c327600e67e","Type":"ContainerStarted","Data":"5a3941a9c362f323175d00938bc64d37605899b0b7125f61838231a86903863a"} Jul 07 19:21:44.120583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:44.120524 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jfpt" podStartSLOduration=2.846428303 podStartE2EDuration="19.120508575s" totalImagesPullingTime="16.274080272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:44.120292546 +0000 UTC m=+19.905277547" watchObservedRunningTime="2026-07-07 19:21:44.120508575 +0000 UTC m=+19.905493575" Jul 07 19:21:45.102576 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:45.102532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"a94452c26ad3b24c9af4c2713dc22706576ac428c29bd6b6758930fd4a56f5e7"} Jul 07 19:21:45.910785 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:45.910750 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:45.910950 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:45.910751 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:45.910950 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:45.910863 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:45.910950 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:45.910751 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:45.911102 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:45.911009 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:45.911102 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:45.911044 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:47.110844 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.110498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" event={"ID":"6dd393af-8584-428f-99b8-2e4fa5f6a86f","Type":"ContainerStarted","Data":"28fec400d6a331928ea17bc1fc0326dd6c01a9896433e7e815c88dc3cd298e26"} Jul 07 19:21:47.110844 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.110825 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:47.112435 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.112407 2580 generic.go:356] "Generic (PLEG): container finished" podID="ceba1e12-77ed-44ff-8d8d-688fba7c2cb8" containerID="7a1c5c005a175e36057664950c45dabfb2b5c52a9356c6ec1b4b8f6f37ff3e34" exitCode=0 Jul 07 19:21:47.112546 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.112465 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerDied","Data":"7a1c5c005a175e36057664950c45dabfb2b5c52a9356c6ec1b4b8f6f37ff3e34"} Jul 07 19:21:47.127899 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.127870 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:47.276383 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.276328 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" podStartSLOduration=6.383351817 podStartE2EDuration="22.276312714s" totalImagesPullingTime="15.892960897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:47.274061511 +0000 UTC m=+23.059046511" watchObservedRunningTime="2026-07-07 19:21:47.276312714 +0000 UTC m=+23.061297738" Jul 07 19:21:47.911109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.911061 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:47.911279 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.911066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:47.911279 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:47.911177 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:47.911279 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:47.911072 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:47.911442 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:47.911284 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:47.911442 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:47.911384 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:48.117394 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:48.117363 2580 generic.go:356] "Generic (PLEG): container finished" podID="ceba1e12-77ed-44ff-8d8d-688fba7c2cb8" containerID="c9951ebb0f5829e1b168da805b40c3b77e7578e1b24a6ce0cf34e5803cc29c71" exitCode=0 Jul 07 19:21:48.117803 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:48.117434 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerDied","Data":"c9951ebb0f5829e1b168da805b40c3b77e7578e1b24a6ce0cf34e5803cc29c71"} Jul 07 19:21:48.118031 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:48.117990 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:48.118134 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:48.118066 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:48.136094 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:48.136065 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:21:49.122574 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:49.122494 2580 generic.go:356] "Generic (PLEG): container finished" podID="ceba1e12-77ed-44ff-8d8d-688fba7c2cb8" containerID="798a5ab857ab59c6df562ecdacd91d52b84d0c879e3d04b44a84f47fa4914f5e" exitCode=0 Jul 07 19:21:49.122923 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:49.122570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerDied","Data":"798a5ab857ab59c6df562ecdacd91d52b84d0c879e3d04b44a84f47fa4914f5e"} Jul 07 19:21:49.911011 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:49.910965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:49.911213 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:49.911018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:49.911213 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:49.911024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:49.911213 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:49.911111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:49.911213 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:49.911184 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:49.911411 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:49.911293 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:51.910727 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:51.910694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:51.911171 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:51.910813 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:51.911171 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:51.911079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:51.911271 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:51.911080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:51.911271 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:51.911189 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:51.911359 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:51.911294 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:53.910333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:53.910300 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:53.910768 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:53.910345 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:53.910768 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:53.910404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:53.910768 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:53.910426 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:53.910768 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:53.910511 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:53.910768 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:53.910616 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:55.911065 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:55.911024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:55.911485 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:55.911024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:55.911485 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:55.911157 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:55.911485 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:55.911025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:55.911485 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:55.911217 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:55.911485 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:55.911304 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:57.145723 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.145544 2580 generic.go:356] "Generic (PLEG): container finished" podID="ceba1e12-77ed-44ff-8d8d-688fba7c2cb8" containerID="674c42dd2ec699470d889a9946b0d3865fa9d50931bb47941ba8f2b0d3b50da4" exitCode=0 Jul 07 19:21:57.146276 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.145613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerDied","Data":"674c42dd2ec699470d889a9946b0d3865fa9d50931bb47941ba8f2b0d3b50da4"} Jul 07 19:21:57.147366 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.147344 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9lg42" event={"ID":"a40d97de-6f3e-45ca-adbc-c4b5acbce37e","Type":"ContainerStarted","Data":"3162fd6025407765137d41d24f8142735b1c8c4471e6bb7915c2be19397e86ab"} Jul 07 19:21:57.196879 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.196825 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9lg42" podStartSLOduration=1.513361049 podStartE2EDuration="32.196799037s" totalImagesPullingTime="30.683437988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-07 19:21:57.196604754 +0000 UTC m=+32.981589776" watchObservedRunningTime="2026-07-07 19:21:57.196799037 +0000 UTC m=+32.981784038" Jul 07 19:21:57.445581 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.445549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:57.445723 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.445661 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:57.445723 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.445712 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret podName:6be412fb-041b-44dc-b9a9-fbd8792bfd19 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:29.445698023 +0000 UTC m=+65.230683014 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret") pod "global-pull-secret-syncer-gs7hm" (UID: "6be412fb-041b-44dc-b9a9-fbd8792bfd19") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:57.546255 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.546224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:57.546415 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.546363 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:57.546500 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.546422 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs podName:9384fe94-f3cc-41ae-a897-53bc166f00b6 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:29.54640812 +0000 UTC m=+65.331393103 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs") pod "network-metrics-daemon-wwmc8" (UID: "9384fe94-f3cc-41ae-a897-53bc166f00b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:57.647090 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.647010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:57.647218 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.647129 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:57.647218 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.647145 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:57.647218 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.647154 2580 projected.go:196] Error preparing data for projected volume kube-api-access-x2grr for pod openshift-network-diagnostics/network-check-target-lks2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:57.647218 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.647201 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr podName:3266e501-d40c-4017-bbc7-bdf001361f1f nodeName:}" failed. No retries permitted until 2026-07-07 19:22:29.647189421 +0000 UTC m=+65.432174399 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2grr" (UniqueName: "kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr") pod "network-check-target-lks2m" (UID: "3266e501-d40c-4017-bbc7-bdf001361f1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:57.910506 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.910420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:57.910673 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.910420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:57.910673 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.910555 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:57.910673 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:57.910420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:57.910673 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.910604 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:57.910673 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:57.910661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:21:58.152482 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:58.152453 2580 generic.go:356] "Generic (PLEG): container finished" podID="ceba1e12-77ed-44ff-8d8d-688fba7c2cb8" containerID="5ae20c73ab9d18b83822419be2f0acd7a15bed88d44176bb3182471b7e2bd799" exitCode=0 Jul 07 19:21:58.152848 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:58.152506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerDied","Data":"5ae20c73ab9d18b83822419be2f0acd7a15bed88d44176bb3182471b7e2bd799"} Jul 07 19:21:59.158315 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:59.158277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gn64" event={"ID":"ceba1e12-77ed-44ff-8d8d-688fba7c2cb8","Type":"ContainerStarted","Data":"8c2d42e218110d1105600d1b679977fea49cfe842f9f5ec0b516281acd32d5ab"} Jul 07 19:21:59.184411 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:59.184359 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6gn64" podStartSLOduration=9.931999348 podStartE2EDuration="34.184345058s" totalImagesPullingTime="24.25234571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:59.182813893 +0000 UTC m=+34.967798894" watchObservedRunningTime="2026-07-07 19:21:59.184345058 +0000 UTC m=+34.969330058" Jul 07 19:21:59.910658 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:59.910623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:21:59.910957 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:59.910720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:21:59.910957 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:59.910724 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:21:59.910957 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:21:59.910772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:21:59.910957 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:59.910873 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:21:59.910957 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:21:59.910953 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:01.910945 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:01.910906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:01.910945 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:01.910942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:01.911436 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:01.910905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:01.911436 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:01.911050 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:01.911436 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:01.911109 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:22:01.911436 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:01.911163 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:22:03.910985 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:03.910957 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:03.911357 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:03.911056 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:03.911357 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:03.911063 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:22:03.911357 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:03.911137 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:03.911357 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:03.911136 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:03.911357 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:03.911207 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:22:05.910234 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:05.910201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:05.910725 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:05.910201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:05.910725 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:05.910308 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:05.910725 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:05.910375 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:22:05.910725 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:05.910201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:05.910725 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:05.910463 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:22:06.925069 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:06.925037 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wwmc8"] Jul 07 19:22:06.925746 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:06.925155 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:06.925746 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:06.925261 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:22:06.927808 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:06.927768 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gs7hm"] Jul 07 19:22:06.927931 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:06.927867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:06.928003 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:06.927965 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:06.928443 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:06.928421 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lks2m"] Jul 07 19:22:06.928568 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:06.928522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:06.928633 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:06.928604 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:22:08.910636 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:08.910427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:08.911113 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:08.910433 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:08.911113 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:08.910722 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:08.911113 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:08.910829 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:22:08.911113 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:08.910439 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:08.911243 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:08.911164 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:22:10.910895 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:10.910861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:10.911289 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:10.910861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:10.911289 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:10.910992 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lks2m" podUID="3266e501-d40c-4017-bbc7-bdf001361f1f" Jul 07 19:22:10.911289 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:10.911093 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwmc8" podUID="9384fe94-f3cc-41ae-a897-53bc166f00b6" Jul 07 19:22:10.911289 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:10.910861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:10.911289 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:10.911192 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gs7hm" podUID="6be412fb-041b-44dc-b9a9-fbd8792bfd19" Jul 07 19:22:11.529269 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.529184 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-242.ec2.internal" event="NodeReady" Jul 07 19:22:11.529420 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.529343 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 19:22:11.600643 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.600610 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hqtsn"] Jul 07 19:22:11.622764 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.622736 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l6tpb"] Jul 07 19:22:11.622937 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.622878 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.626459 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.626433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dwrjw\"" Jul 07 19:22:11.626589 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.626433 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 19:22:11.626640 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.626597 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 19:22:11.626640 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.626622 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 19:22:11.642878 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.642853 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqtsn"] Jul 07 19:22:11.643020 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.642892 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l6tpb"] Jul 07 19:22:11.643020 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.642991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.647252 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4866e6-d381-44d0-9238-448b1d785623-config-volume\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.647374 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lxjvf\"" Jul 07 19:22:11.647374 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b4866e6-d381-44d0-9238-448b1d785623-metrics-tls\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.647374 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4866e6-d381-44d0-9238-448b1d785623-tmp-dir\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.647545 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66ph\" (UniqueName: \"kubernetes.io/projected/6b4866e6-d381-44d0-9238-448b1d785623-kube-api-access-h66ph\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.647545 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489936c9-f207-48f5-9a80-a171c4be51aa-cert\") pod \"ingress-canary-hqtsn\" (UID: \"489936c9-f207-48f5-9a80-a171c4be51aa\") " pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.647545 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 19:22:11.647545 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647520 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 19:22:11.647545 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.647525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhkkt\" (UniqueName: \"kubernetes.io/projected/489936c9-f207-48f5-9a80-a171c4be51aa-kube-api-access-fhkkt\") pod \"ingress-canary-hqtsn\" (UID: \"489936c9-f207-48f5-9a80-a171c4be51aa\") " pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.655699 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.655674 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m9dvd"] Jul 07 19:22:11.670306 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.670286 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.673798 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.673462 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 19:22:11.673798 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.673554 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 19:22:11.676908 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.674236 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 19:22:11.676908 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.674245 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 19:22:11.676908 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.674339 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ktbrl\"" Jul 07 19:22:11.677489 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.677462 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m9dvd"] Jul 07 19:22:11.748427 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489936c9-f207-48f5-9a80-a171c4be51aa-cert\") pod \"ingress-canary-hqtsn\" (UID: \"489936c9-f207-48f5-9a80-a171c4be51aa\") " pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.748427 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v7p2\" (UniqueName: \"kubernetes.io/projected/0f5670ee-b31c-4476-89b5-f8dac251fb24-kube-api-access-9v7p2\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhkkt\" (UniqueName: \"kubernetes.io/projected/489936c9-f207-48f5-9a80-a171c4be51aa-kube-api-access-fhkkt\") pod \"ingress-canary-hqtsn\" (UID: \"489936c9-f207-48f5-9a80-a171c4be51aa\") " pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748495 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4866e6-d381-44d0-9238-448b1d785623-config-volume\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f5670ee-b31c-4476-89b5-f8dac251fb24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f5670ee-b31c-4476-89b5-f8dac251fb24-crio-socket\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f5670ee-b31c-4476-89b5-f8dac251fb24-data-volume\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b4866e6-d381-44d0-9238-448b1d785623-metrics-tls\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.748677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4866e6-d381-44d0-9238-448b1d785623-tmp-dir\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.749034 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66ph\" (UniqueName: \"kubernetes.io/projected/6b4866e6-d381-44d0-9238-448b1d785623-kube-api-access-h66ph\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.749034 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.748720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f5670ee-b31c-4476-89b5-f8dac251fb24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.749034 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.749010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4866e6-d381-44d0-9238-448b1d785623-tmp-dir\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.749200 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.749181 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4866e6-d381-44d0-9238-448b1d785623-config-volume\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.752523 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.752496 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489936c9-f207-48f5-9a80-a171c4be51aa-cert\") pod \"ingress-canary-hqtsn\" (UID: \"489936c9-f207-48f5-9a80-a171c4be51aa\") " pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.752619 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.752500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b4866e6-d381-44d0-9238-448b1d785623-metrics-tls\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.766846 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.766818 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66ph\" (UniqueName: \"kubernetes.io/projected/6b4866e6-d381-44d0-9238-448b1d785623-kube-api-access-h66ph\") pod \"dns-default-l6tpb\" (UID: \"6b4866e6-d381-44d0-9238-448b1d785623\") " pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.769399 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.769375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhkkt\" (UniqueName: \"kubernetes.io/projected/489936c9-f207-48f5-9a80-a171c4be51aa-kube-api-access-fhkkt\") pod \"ingress-canary-hqtsn\" (UID: \"489936c9-f207-48f5-9a80-a171c4be51aa\") " pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.849318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849278 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v7p2\" (UniqueName: \"kubernetes.io/projected/0f5670ee-b31c-4476-89b5-f8dac251fb24-kube-api-access-9v7p2\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.849494 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f5670ee-b31c-4476-89b5-f8dac251fb24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.849494 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849388 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f5670ee-b31c-4476-89b5-f8dac251fb24-crio-socket\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.849494 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f5670ee-b31c-4476-89b5-f8dac251fb24-data-volume\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.849494 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f5670ee-b31c-4476-89b5-f8dac251fb24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.849710 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849602 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f5670ee-b31c-4476-89b5-f8dac251fb24-crio-socket\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.849867 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849838 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f5670ee-b31c-4476-89b5-f8dac251fb24-data-volume\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.850013 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.849965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f5670ee-b31c-4476-89b5-f8dac251fb24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.851735 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.851714 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f5670ee-b31c-4476-89b5-f8dac251fb24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.858034 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.858014 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v7p2\" (UniqueName: \"kubernetes.io/projected/0f5670ee-b31c-4476-89b5-f8dac251fb24-kube-api-access-9v7p2\") pod \"insights-runtime-extractor-m9dvd\" (UID: \"0f5670ee-b31c-4476-89b5-f8dac251fb24\") " pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:11.932660 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.932630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqtsn" Jul 07 19:22:11.951059 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.951025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:11.983500 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:11.983477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m9dvd" Jul 07 19:22:12.148048 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.148016 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqtsn"] Jul 07 19:22:12.151244 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:12.151218 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489936c9_f207_48f5_9a80_a171c4be51aa.slice/crio-00ea954779515cd0a11e4f6694b221a0c3814840f33e2928f814186a10d71bba WatchSource:0}: Error finding container 00ea954779515cd0a11e4f6694b221a0c3814840f33e2928f814186a10d71bba: Status 404 returned error can't find the container with id 00ea954779515cd0a11e4f6694b221a0c3814840f33e2928f814186a10d71bba Jul 07 19:22:12.188490 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.188459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqtsn" event={"ID":"489936c9-f207-48f5-9a80-a171c4be51aa","Type":"ContainerStarted","Data":"00ea954779515cd0a11e4f6694b221a0c3814840f33e2928f814186a10d71bba"} Jul 07 19:22:12.268196 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.268161 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fb9c47d5d-j6ls8"] Jul 07 19:22:12.287395 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.287369 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb9c47d5d-j6ls8"] Jul 07 19:22:12.287557 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.287462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.290692 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.290670 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 19:22:12.290692 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.290688 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 19:22:12.290883 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.290732 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 19:22:12.290883 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.290758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 19:22:12.290883 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.290754 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pj6kk\"" Jul 07 19:22:12.291117 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.291102 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 19:22:12.291375 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.291360 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 19:22:12.291440 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.291389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 19:22:12.353920 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.353853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-serving-cert\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.353920 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.353884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-service-ca\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.353920 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.353911 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjlpl\" (UniqueName: \"kubernetes.io/projected/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-kube-api-access-zjlpl\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.354141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.353942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-oauth-serving-cert\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.354141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.354032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-oauth-config\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.354141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.354082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-config\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.372247 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.372208 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l6tpb"] Jul 07 19:22:12.373361 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.373015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m9dvd"] Jul 07 19:22:12.376659 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:12.376632 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f5670ee_b31c_4476_89b5_f8dac251fb24.slice/crio-5b75afe7febed908d13414f5e4a01516db4379d7c5c8759ad248188223cc1d6d WatchSource:0}: Error finding container 5b75afe7febed908d13414f5e4a01516db4379d7c5c8759ad248188223cc1d6d: Status 404 returned error can't find the container with id 5b75afe7febed908d13414f5e4a01516db4379d7c5c8759ad248188223cc1d6d Jul 07 19:22:12.377995 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:12.377952 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4866e6_d381_44d0_9238_448b1d785623.slice/crio-21b1931af147eb3f9a4e3e5a3bed852e767ad3e462f0cbe607a63c6332797e30 WatchSource:0}: Error finding container 21b1931af147eb3f9a4e3e5a3bed852e767ad3e462f0cbe607a63c6332797e30: Status 404 returned error can't find the container with id 21b1931af147eb3f9a4e3e5a3bed852e767ad3e462f0cbe607a63c6332797e30 Jul 07 19:22:12.455400 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.455355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-oauth-serving-cert\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.455543 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.455408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-oauth-config\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.455543 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.455446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-config\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.455543 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.455491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-serving-cert\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.455689 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.455621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-service-ca\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.455742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.455693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjlpl\" (UniqueName: \"kubernetes.io/projected/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-kube-api-access-zjlpl\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.456178 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.456158 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-config\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.456289 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.456163 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-oauth-serving-cert\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.456289 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.456246 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-service-ca\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.467630 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.467610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-serving-cert\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.467630 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.467625 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-oauth-config\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.467790 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.467775 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjlpl\" (UniqueName: \"kubernetes.io/projected/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-kube-api-access-zjlpl\") pod \"console-5fb9c47d5d-j6ls8\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.597080 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.597016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:12.743000 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.742890 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb9c47d5d-j6ls8"] Jul 07 19:22:12.747662 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:12.747627 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13e8cf2_a9ef_4eef_86d3_2f008a75e44e.slice/crio-64f05107a0e6073722f901cc729dc4934695a2be40be52fc1c9cce353cf44af7 WatchSource:0}: Error finding container 64f05107a0e6073722f901cc729dc4934695a2be40be52fc1c9cce353cf44af7: Status 404 returned error can't find the container with id 64f05107a0e6073722f901cc729dc4934695a2be40be52fc1c9cce353cf44af7 Jul 07 19:22:12.910880 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.910805 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:12.911062 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.910804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:12.911062 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.911001 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:12.914079 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.913873 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 19:22:12.914079 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.913884 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 19:22:12.914694 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.914673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 19:22:12.914803 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.914717 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 19:22:12.914878 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.914843 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rq87d\"" Jul 07 19:22:12.914935 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:12.914872 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhh8r\"" Jul 07 19:22:13.194017 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:13.193760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l6tpb" event={"ID":"6b4866e6-d381-44d0-9238-448b1d785623","Type":"ContainerStarted","Data":"21b1931af147eb3f9a4e3e5a3bed852e767ad3e462f0cbe607a63c6332797e30"} Jul 07 19:22:13.196271 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:13.196240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb9c47d5d-j6ls8" event={"ID":"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e","Type":"ContainerStarted","Data":"64f05107a0e6073722f901cc729dc4934695a2be40be52fc1c9cce353cf44af7"} Jul 07 19:22:13.198807 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:13.198737 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9dvd" event={"ID":"0f5670ee-b31c-4476-89b5-f8dac251fb24","Type":"ContainerStarted","Data":"5b75afe7febed908d13414f5e4a01516db4379d7c5c8759ad248188223cc1d6d"} Jul 07 19:22:14.238003 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.237913 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79747b9866-pjgbn"] Jul 07 19:22:14.247923 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.247867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.258070 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.257871 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79747b9866-pjgbn"] Jul 07 19:22:14.259954 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.259135 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 19:22:14.271294 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-trusted-ca-bundle\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.271465 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxl4l\" (UniqueName: \"kubernetes.io/projected/61f914ed-895d-44b9-a8fc-0b516137c0eb-kube-api-access-dxl4l\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.271465 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-serving-cert\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.271465 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-service-ca\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.271465 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271409 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-oauth-config\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.271677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-config\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.271677 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.271539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-oauth-serving-cert\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372202 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-oauth-serving-cert\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372395 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-trusted-ca-bundle\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372557 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxl4l\" (UniqueName: \"kubernetes.io/projected/61f914ed-895d-44b9-a8fc-0b516137c0eb-kube-api-access-dxl4l\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372646 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-serving-cert\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372646 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-service-ca\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372749 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-oauth-config\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.372749 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-config\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.373010 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.372936 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-oauth-serving-cert\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.373259 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.373215 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-trusted-ca-bundle\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.373426 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.373362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-service-ca\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.373519 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.373438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-config\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.376074 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.375676 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-oauth-config\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.376181 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.376146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-serving-cert\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.384152 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.384099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxl4l\" (UniqueName: \"kubernetes.io/projected/61f914ed-895d-44b9-a8fc-0b516137c0eb-kube-api-access-dxl4l\") pod \"console-79747b9866-pjgbn\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:14.563891 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:14.563753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:18.714630 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.714592 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws"] Jul 07 19:22:18.723784 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.723759 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.740692 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.740668 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 19:22:18.740692 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.740684 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 19:22:18.740953 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.740737 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 19:22:18.740953 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.740891 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 19:22:18.741214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.741188 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 19:22:18.741300 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.741285 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7wvfq\"" Jul 07 19:22:18.827449 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.827324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fe63467-dcd8-4210-bcf5-8da024890cbe-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.827645 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.827456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fe63467-dcd8-4210-bcf5-8da024890cbe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.827645 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.827496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gspg8\" (UniqueName: \"kubernetes.io/projected/1fe63467-dcd8-4210-bcf5-8da024890cbe-kube-api-access-gspg8\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.827645 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.827525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fe63467-dcd8-4210-bcf5-8da024890cbe-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.861588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.861555 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws"] Jul 07 19:22:18.928363 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.928330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gspg8\" (UniqueName: \"kubernetes.io/projected/1fe63467-dcd8-4210-bcf5-8da024890cbe-kube-api-access-gspg8\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.928363 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.928366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fe63467-dcd8-4210-bcf5-8da024890cbe-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.928602 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.928461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fe63467-dcd8-4210-bcf5-8da024890cbe-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.928602 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.928531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fe63467-dcd8-4210-bcf5-8da024890cbe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.929201 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.929176 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fe63467-dcd8-4210-bcf5-8da024890cbe-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.931132 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.931106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fe63467-dcd8-4210-bcf5-8da024890cbe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.931238 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.931217 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fe63467-dcd8-4210-bcf5-8da024890cbe-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:18.946095 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:18.946071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gspg8\" (UniqueName: \"kubernetes.io/projected/1fe63467-dcd8-4210-bcf5-8da024890cbe-kube-api-access-gspg8\") pod \"prometheus-operator-567f7b8b7c-g28ws\" (UID: \"1fe63467-dcd8-4210-bcf5-8da024890cbe\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:19.034132 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:19.034051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" Jul 07 19:22:19.323887 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:19.315519 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79747b9866-pjgbn"] Jul 07 19:22:19.323887 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:19.322963 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f914ed_895d_44b9_a8fc_0b516137c0eb.slice/crio-5d0347d06c72897fca387335f1484ceee6fc360d1062c1e248346b7209f0f311 WatchSource:0}: Error finding container 5d0347d06c72897fca387335f1484ceee6fc360d1062c1e248346b7209f0f311: Status 404 returned error can't find the container with id 5d0347d06c72897fca387335f1484ceee6fc360d1062c1e248346b7209f0f311 Jul 07 19:22:19.402706 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:19.402428 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe63467_dcd8_4210_bcf5_8da024890cbe.slice/crio-bdc01046cb7caf9d897eb437e2c209a43d0ba3977f7f67f6191d17e5bba9887e WatchSource:0}: Error finding container bdc01046cb7caf9d897eb437e2c209a43d0ba3977f7f67f6191d17e5bba9887e: Status 404 returned error can't find the container with id bdc01046cb7caf9d897eb437e2c209a43d0ba3977f7f67f6191d17e5bba9887e Jul 07 19:22:19.406961 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:19.406935 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws"] Jul 07 19:22:20.137426 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.137396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dmrlr" Jul 07 19:22:20.228321 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.228231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb9c47d5d-j6ls8" event={"ID":"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e","Type":"ContainerStarted","Data":"ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447"} Jul 07 19:22:20.229726 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.229701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqtsn" event={"ID":"489936c9-f207-48f5-9a80-a171c4be51aa","Type":"ContainerStarted","Data":"bbd4dac246b1273909d1baa4b76df2ed62129cbc70e3caf533980750c40b10ef"} Jul 07 19:22:20.231372 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.231350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9dvd" event={"ID":"0f5670ee-b31c-4476-89b5-f8dac251fb24","Type":"ContainerStarted","Data":"38c5ca338af08073d243acab247f68a29ebe906f4cd70bded4537de1b651d26b"} Jul 07 19:22:20.231463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.231395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9dvd" event={"ID":"0f5670ee-b31c-4476-89b5-f8dac251fb24","Type":"ContainerStarted","Data":"fca6393dbd159575f664fada98512fd89dab9acff0eb9d077812ed4885120c43"} Jul 07 19:22:20.232922 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.232888 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l6tpb" event={"ID":"6b4866e6-d381-44d0-9238-448b1d785623","Type":"ContainerStarted","Data":"77e237b5cec31436a540adc4c766840261e9b76ce45c1da746bb5a7dfcfbd034"} Jul 07 19:22:20.233022 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.232943 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l6tpb" event={"ID":"6b4866e6-d381-44d0-9238-448b1d785623","Type":"ContainerStarted","Data":"51619c7558dac9a2dd64d1d2eacfbccbb4556a915042287968415953a22ce66d"} Jul 07 19:22:20.233022 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.232993 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:20.234252 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.234231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79747b9866-pjgbn" event={"ID":"61f914ed-895d-44b9-a8fc-0b516137c0eb","Type":"ContainerStarted","Data":"e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1"} Jul 07 19:22:20.234352 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.234269 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79747b9866-pjgbn" event={"ID":"61f914ed-895d-44b9-a8fc-0b516137c0eb","Type":"ContainerStarted","Data":"5d0347d06c72897fca387335f1484ceee6fc360d1062c1e248346b7209f0f311"} Jul 07 19:22:20.235244 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.235225 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" event={"ID":"1fe63467-dcd8-4210-bcf5-8da024890cbe","Type":"ContainerStarted","Data":"bdc01046cb7caf9d897eb437e2c209a43d0ba3977f7f67f6191d17e5bba9887e"} Jul 07 19:22:20.314693 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.314644 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fb9c47d5d-j6ls8" podStartSLOduration=1.928029117 podStartE2EDuration="8.314622427s" totalImagesPullingTime="6.38659331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:20.283904796 +0000 UTC m=+56.068889797" watchObservedRunningTime="2026-07-07 19:22:20.314622427 +0000 UTC m=+56.099607428" Jul 07 19:22:20.351803 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.351750 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l6tpb" podStartSLOduration=2.601890922 podStartE2EDuration="9.351706551s" totalImagesPullingTime="6.749815629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:20.314191701 +0000 UTC m=+56.099176702" watchObservedRunningTime="2026-07-07 19:22:20.351706551 +0000 UTC m=+56.136691555" Jul 07 19:22:20.352045 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:20.351844 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79747b9866-pjgbn" podStartSLOduration=6.351838278 podStartE2EDuration="6.351838278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:20.350920953 +0000 UTC m=+56.135905956" watchObservedRunningTime="2026-07-07 19:22:20.351838278 +0000 UTC m=+56.136823277" Jul 07 19:22:21.241450 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:21.241409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" event={"ID":"1fe63467-dcd8-4210-bcf5-8da024890cbe","Type":"ContainerStarted","Data":"4093bbad083d2ad2267ff39b08e03e2b1ea3b0fce10ff615a7b70bcc5b115017"} Jul 07 19:22:21.241903 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:21.241470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" event={"ID":"1fe63467-dcd8-4210-bcf5-8da024890cbe","Type":"ContainerStarted","Data":"e5e91dd827cb1f7eca47b504ec26a449189b1a817e30d1ca57a429ec04cfb4d4"} Jul 07 19:22:21.267171 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:21.267110 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-g28ws" podStartSLOduration=2.053788344 podStartE2EDuration="3.267089579s" totalImagesPullingTime="1.213301235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:21.265909719 +0000 UTC m=+57.050894720" watchObservedRunningTime="2026-07-07 19:22:21.267089579 +0000 UTC m=+57.052074580" Jul 07 19:22:21.267555 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:21.267524 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hqtsn" podStartSLOduration=3.290681125 podStartE2EDuration="10.267517536s" totalImagesPullingTime="6.976836411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:20.38004384 +0000 UTC m=+56.165028840" watchObservedRunningTime="2026-07-07 19:22:21.267517536 +0000 UTC m=+57.052502519" Jul 07 19:22:22.598141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:22.598102 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:22.598590 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:22.598205 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:22.599893 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:22.599853 2580 patch_prober.go:28] interesting pod/console-5fb9c47d5d-j6ls8 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.9:8443/health\": dial tcp 10.133.0.9:8443: connect: connection refused" start-of-body= Jul 07 19:22:22.600061 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:22.599908 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5fb9c47d5d-j6ls8" podUID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" containerName="console" probeResult="failure" output="Get \"https://10.133.0.9:8443/health\": dial tcp 10.133.0.9:8443: connect: connection refused" Jul 07 19:22:23.092468 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.092433 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rk8v4"] Jul 07 19:22:23.108654 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.108625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.110857 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.110831 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 19:22:23.111161 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.111096 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 19:22:23.111161 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.111140 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tcdnl\"" Jul 07 19:22:23.111669 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.111655 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 19:22:23.165035 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-root\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165221 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165221 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x94sb\" (UniqueName: \"kubernetes.io/projected/52aa9653-287d-4042-a1b3-49ce179e85e8-kube-api-access-x94sb\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165221 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-sys\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165395 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-textfile\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165395 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-wtmp\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165395 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52aa9653-287d-4042-a1b3-49ce179e85e8-metrics-client-ca\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165395 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-tls\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.165558 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.165452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.266623 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.266623 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-root\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.266856 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-root\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.266856 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.266856 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x94sb\" (UniqueName: \"kubernetes.io/projected/52aa9653-287d-4042-a1b3-49ce179e85e8-kube-api-access-x94sb\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.266856 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-sys\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267083 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-textfile\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267083 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266917 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-sys\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267083 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-wtmp\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267083 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.266990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52aa9653-287d-4042-a1b3-49ce179e85e8-metrics-client-ca\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267083 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.267021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-tls\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267252 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.267113 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-wtmp\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267252 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:23.267153 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 19:22:23.267252 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:23.267231 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-tls podName:52aa9653-287d-4042-a1b3-49ce179e85e8 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:23.767212508 +0000 UTC m=+59.552197505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-tls") pod "node-exporter-rk8v4" (UID: "52aa9653-287d-4042-a1b3-49ce179e85e8") : secret "node-exporter-tls" not found Jul 07 19:22:23.267451 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.267430 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267484 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.267462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-textfile\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.267514 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.267487 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52aa9653-287d-4042-a1b3-49ce179e85e8-metrics-client-ca\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.269047 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.269030 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.276133 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.276110 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x94sb\" (UniqueName: \"kubernetes.io/projected/52aa9653-287d-4042-a1b3-49ce179e85e8-kube-api-access-x94sb\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.772391 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.772352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-tls\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:23.774683 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:23.774661 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa9653-287d-4042-a1b3-49ce179e85e8-node-exporter-tls\") pod \"node-exporter-rk8v4\" (UID: \"52aa9653-287d-4042-a1b3-49ce179e85e8\") " pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:24.017576 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.017540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rk8v4" Jul 07 19:22:24.047966 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:24.047932 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52aa9653_287d_4042_a1b3_49ce179e85e8.slice/crio-5e9fe2787cdb225038c70fd0121da171589c03366a82c0b1bc548311404c7b6e WatchSource:0}: Error finding container 5e9fe2787cdb225038c70fd0121da171589c03366a82c0b1bc548311404c7b6e: Status 404 returned error can't find the container with id 5e9fe2787cdb225038c70fd0121da171589c03366a82c0b1bc548311404c7b6e Jul 07 19:22:24.255562 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.255525 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9dvd" event={"ID":"0f5670ee-b31c-4476-89b5-f8dac251fb24","Type":"ContainerStarted","Data":"d9dd3ffa87dfd14ffdb8c086ee310cf19c62b32883afe3042715fdcba4371a08"} Jul 07 19:22:24.256643 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.256612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rk8v4" event={"ID":"52aa9653-287d-4042-a1b3-49ce179e85e8","Type":"ContainerStarted","Data":"5e9fe2787cdb225038c70fd0121da171589c03366a82c0b1bc548311404c7b6e"} Jul 07 19:22:24.564937 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.564894 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:24.565143 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.564991 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:24.569738 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.569712 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:24.596750 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:24.596700 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m9dvd" podStartSLOduration=2.970616885 podStartE2EDuration="13.596681603s" totalImagesPullingTime="10.626064718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:22:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:24.279425552 +0000 UTC m=+60.064410554" watchObservedRunningTime="2026-07-07 19:22:24.596681603 +0000 UTC m=+60.381666603" Jul 07 19:22:25.171404 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.171368 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c495498f6-wc5kj"] Jul 07 19:22:25.187062 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.187024 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c495498f6-wc5kj"] Jul 07 19:22:25.187225 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.187159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.190077 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.190055 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 19:22:25.190211 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.190074 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 19:22:25.190277 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.190225 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cnl4fhb00gqeq\"" Jul 07 19:22:25.190277 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.190251 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 19:22:25.190379 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.190301 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 19:22:25.190636 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.190619 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 19:22:25.191045 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.191030 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ftqq6\"" Jul 07 19:22:25.263299 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.263270 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:22:25.286895 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.286862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.286895 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.286894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.287109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.286957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-grpc-tls\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.287109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.287006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.287109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.287029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.287109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.287067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cab8f4d-b087-434f-a4c5-41afd5161c63-metrics-client-ca\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.287109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.287106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-tls\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.287256 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.287132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr2w2\" (UniqueName: \"kubernetes.io/projected/2cab8f4d-b087-434f-a4c5-41afd5161c63-kube-api-access-tr2w2\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.313156 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.313129 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb9c47d5d-j6ls8"] Jul 07 19:22:25.388524 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.388485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.388719 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.388533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.388719 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.388597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-grpc-tls\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.388719 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.388618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.388719 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.388646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.388930 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.388906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cab8f4d-b087-434f-a4c5-41afd5161c63-metrics-client-ca\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.389046 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.389020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-tls\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.389950 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.389127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr2w2\" (UniqueName: \"kubernetes.io/projected/2cab8f4d-b087-434f-a4c5-41afd5161c63-kube-api-access-tr2w2\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.389950 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.389602 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cab8f4d-b087-434f-a4c5-41afd5161c63-metrics-client-ca\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.391281 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.391255 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.391830 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.391802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.391920 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.391833 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.391993 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.391920 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.392106 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.392084 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-grpc-tls\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.392161 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.392125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2cab8f4d-b087-434f-a4c5-41afd5161c63-secret-thanos-querier-tls\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.399056 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.399027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr2w2\" (UniqueName: \"kubernetes.io/projected/2cab8f4d-b087-434f-a4c5-41afd5161c63-kube-api-access-tr2w2\") pod \"thanos-querier-6c495498f6-wc5kj\" (UID: \"2cab8f4d-b087-434f-a4c5-41afd5161c63\") " pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.497233 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.497153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:25.700472 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:25.700447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c495498f6-wc5kj"] Jul 07 19:22:25.702480 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:25.702453 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cab8f4d_b087_434f_a4c5_41afd5161c63.slice/crio-44984e5ac222af99882b6fc0c4664bcb4a4af4fb0c2277d75bf99972e9cb216f WatchSource:0}: Error finding container 44984e5ac222af99882b6fc0c4664bcb4a4af4fb0c2277d75bf99972e9cb216f: Status 404 returned error can't find the container with id 44984e5ac222af99882b6fc0c4664bcb4a4af4fb0c2277d75bf99972e9cb216f Jul 07 19:22:26.266630 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:26.266543 2580 generic.go:356] "Generic (PLEG): container finished" podID="52aa9653-287d-4042-a1b3-49ce179e85e8" containerID="bca2dd8a255ce9032b1a4ab5eab3cdfa7c91a8c9e450212c89f0edbf359cfc70" exitCode=0 Jul 07 19:22:26.266630 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:26.266589 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rk8v4" event={"ID":"52aa9653-287d-4042-a1b3-49ce179e85e8","Type":"ContainerDied","Data":"bca2dd8a255ce9032b1a4ab5eab3cdfa7c91a8c9e450212c89f0edbf359cfc70"} Jul 07 19:22:26.267988 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:26.267942 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"44984e5ac222af99882b6fc0c4664bcb4a4af4fb0c2277d75bf99972e9cb216f"} Jul 07 19:22:27.274463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.274428 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rk8v4" event={"ID":"52aa9653-287d-4042-a1b3-49ce179e85e8","Type":"ContainerStarted","Data":"5592363750a34e8906aa801b147d1b8d333addcaa75f16f7452f941f7dcc3e92"} Jul 07 19:22:27.274892 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.274492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rk8v4" event={"ID":"52aa9653-287d-4042-a1b3-49ce179e85e8","Type":"ContainerStarted","Data":"06afb39f953aaaaa8c3dd0d81fea89d0caa775354dd8563a9a2428fcb12251f2"} Jul 07 19:22:27.296547 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.296487 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rk8v4" podStartSLOduration=2.6868804170000002 podStartE2EDuration="4.296464605s" totalImagesPullingTime="1.511287382s" totalInitContainerRuntime="98.296806ms" isStatefulPod=true podCreationTimestamp="2026-07-07 19:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:27.295274752 +0000 UTC m=+63.080259764" watchObservedRunningTime="2026-07-07 19:22:27.296464605 +0000 UTC m=+63.081449606" Jul 07 19:22:27.860277 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.860243 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9"] Jul 07 19:22:27.874872 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.874845 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9"] Jul 07 19:22:27.875084 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.874934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:27.877657 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.877633 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 19:22:27.877950 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:27.877927 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xx8zq\"" Jul 07 19:22:28.010945 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.010910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwxm\" (UniqueName: \"kubernetes.io/projected/286e1d46-4c13-4955-a6a7-bd623500eedd-kube-api-access-rhwxm\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.011124 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.010955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/286e1d46-4c13-4955-a6a7-bd623500eedd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.112096 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.112045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwxm\" (UniqueName: \"kubernetes.io/projected/286e1d46-4c13-4955-a6a7-bd623500eedd-kube-api-access-rhwxm\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.112096 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.112083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/286e1d46-4c13-4955-a6a7-bd623500eedd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.112227 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:28.112214 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 19:22:28.112275 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:28.112266 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/286e1d46-4c13-4955-a6a7-bd623500eedd-monitoring-plugin-cert podName:286e1d46-4c13-4955-a6a7-bd623500eedd nodeName:}" failed. No retries permitted until 2026-07-07 19:22:28.612252559 +0000 UTC m=+64.397237540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/286e1d46-4c13-4955-a6a7-bd623500eedd-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-qvqs9" (UID: "286e1d46-4c13-4955-a6a7-bd623500eedd") : secret "monitoring-plugin-cert" not found Jul 07 19:22:28.121702 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.121675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwxm\" (UniqueName: \"kubernetes.io/projected/286e1d46-4c13-4955-a6a7-bd623500eedd-kube-api-access-rhwxm\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.279365 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.279327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"2db5cec2ae4c66196dff19bfe1d061a7d32df5ffae4c3a9d33b78b7ed3c0e835"} Jul 07 19:22:28.617224 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.617192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/286e1d46-4c13-4955-a6a7-bd623500eedd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.619543 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.619512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/286e1d46-4c13-4955-a6a7-bd623500eedd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qvqs9\" (UID: \"286e1d46-4c13-4955-a6a7-bd623500eedd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.784753 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.784655 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:28.841695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.841663 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b4b7fd877-nkhx8"] Jul 07 19:22:28.863172 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.863136 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b4b7fd877-nkhx8"] Jul 07 19:22:28.863343 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.863266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:28.946496 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:28.946466 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9"] Jul 07 19:22:28.949209 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:28.949178 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286e1d46_4c13_4955_a6a7_bd623500eedd.slice/crio-b3b0b61aa12e3c0252751cadc09f40ca908d0bb84e0047108e01adbcc40b310a WatchSource:0}: Error finding container b3b0b61aa12e3c0252751cadc09f40ca908d0bb84e0047108e01adbcc40b310a: Status 404 returned error can't find the container with id b3b0b61aa12e3c0252751cadc09f40ca908d0bb84e0047108e01adbcc40b310a Jul 07 19:22:29.019297 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-oauth-config\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.019297 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019303 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-oauth-serving-cert\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.019551 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-serving-cert\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.019551 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc8bz\" (UniqueName: \"kubernetes.io/projected/5ff2082c-f500-43b6-b11f-bab5f407170f-kube-api-access-gc8bz\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.019551 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-service-ca\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.019671 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-console-config\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.019671 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.019637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-trusted-ca-bundle\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.120907 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.120864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-oauth-config\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.120907 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.120914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-oauth-serving-cert\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.121188 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.121028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-serving-cert\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.121188 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.121072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc8bz\" (UniqueName: \"kubernetes.io/projected/5ff2082c-f500-43b6-b11f-bab5f407170f-kube-api-access-gc8bz\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.121188 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.121100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-service-ca\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.121188 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.121131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-console-config\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.121188 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.121157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-trusted-ca-bundle\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.122679 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.122220 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-trusted-ca-bundle\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.123155 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.123118 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-service-ca\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.123710 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.123688 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-console-config\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.124169 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.124129 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-oauth-serving-cert\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.126078 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.126054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-serving-cert\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.126253 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.126234 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-oauth-config\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.131856 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.131836 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc8bz\" (UniqueName: \"kubernetes.io/projected/5ff2082c-f500-43b6-b11f-bab5f407170f-kube-api-access-gc8bz\") pod \"console-6b4b7fd877-nkhx8\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.173176 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.173138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:29.294648 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.294611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"4da0b959c9be9c90d4e8b0c787d6a3c1e2f901ddba542e1d3f33bc52081f9759"} Jul 07 19:22:29.295127 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.294787 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"a447c9f94d7c5eab0763d40ca1e933217469d3cf220897ece534f18e2309abd8"} Jul 07 19:22:29.296239 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.296202 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" event={"ID":"286e1d46-4c13-4955-a6a7-bd623500eedd","Type":"ContainerStarted","Data":"b3b0b61aa12e3c0252751cadc09f40ca908d0bb84e0047108e01adbcc40b310a"} Jul 07 19:22:29.331051 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.330951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b4b7fd877-nkhx8"] Jul 07 19:22:29.459335 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:29.459255 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff2082c_f500_43b6_b11f_bab5f407170f.slice/crio-b96c994b74049f260eb7e68be7cfa0b3838e6073784b38c7c8bdd26652914438 WatchSource:0}: Error finding container b96c994b74049f260eb7e68be7cfa0b3838e6073784b38c7c8bdd26652914438: Status 404 returned error can't find the container with id b96c994b74049f260eb7e68be7cfa0b3838e6073784b38c7c8bdd26652914438 Jul 07 19:22:29.526325 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.526296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:29.530690 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.530669 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 19:22:29.539499 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.539389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be412fb-041b-44dc-b9a9-fbd8792bfd19-original-pull-secret\") pod \"global-pull-secret-syncer-gs7hm\" (UID: \"6be412fb-041b-44dc-b9a9-fbd8792bfd19\") " pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:29.627554 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.627509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:29.630549 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.630519 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 19:22:29.641352 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.641317 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9384fe94-f3cc-41ae-a897-53bc166f00b6-metrics-certs\") pod \"network-metrics-daemon-wwmc8\" (UID: \"9384fe94-f3cc-41ae-a897-53bc166f00b6\") " pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:29.726455 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.726386 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rq87d\"" Jul 07 19:22:29.728214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.728186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:29.730294 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.730277 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 19:22:29.730394 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.730350 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gs7hm" Jul 07 19:22:29.735091 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.735074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwmc8" Jul 07 19:22:29.741419 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.741402 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 19:22:29.752858 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:29.752833 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2grr\" (UniqueName: \"kubernetes.io/projected/3266e501-d40c-4017-bbc7-bdf001361f1f-kube-api-access-x2grr\") pod \"network-check-target-lks2m\" (UID: \"3266e501-d40c-4017-bbc7-bdf001361f1f\") " pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:30.004134 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.004059 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gs7hm"] Jul 07 19:22:30.007760 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:30.007731 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be412fb_041b_44dc_b9a9_fbd8792bfd19.slice/crio-7cbe4aef7dd6fe1b7e603be875feec000af2b983ffd8995f1dfca7a33f9a9a60 WatchSource:0}: Error finding container 7cbe4aef7dd6fe1b7e603be875feec000af2b983ffd8995f1dfca7a33f9a9a60: Status 404 returned error can't find the container with id 7cbe4aef7dd6fe1b7e603be875feec000af2b983ffd8995f1dfca7a33f9a9a60 Jul 07 19:22:30.024808 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.024780 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wwmc8"] Jul 07 19:22:30.028463 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:30.028436 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9384fe94_f3cc_41ae_a897_53bc166f00b6.slice/crio-0f99383171277cf786bbb057fcdebfa5220396d60d6976c7f08ae6b018e1942e WatchSource:0}: Error finding container 0f99383171277cf786bbb057fcdebfa5220396d60d6976c7f08ae6b018e1942e: Status 404 returned error can't find the container with id 0f99383171277cf786bbb057fcdebfa5220396d60d6976c7f08ae6b018e1942e Jul 07 19:22:30.039312 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.039292 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhh8r\"" Jul 07 19:22:30.048342 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.048314 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:30.220948 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.220917 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lks2m"] Jul 07 19:22:30.225377 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:30.225346 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3266e501_d40c_4017_bbc7_bdf001361f1f.slice/crio-b868b9ef0ca1fa40629cd52dcb4127b44306780ee0cc04e169929a5b43b70d40 WatchSource:0}: Error finding container b868b9ef0ca1fa40629cd52dcb4127b44306780ee0cc04e169929a5b43b70d40: Status 404 returned error can't find the container with id b868b9ef0ca1fa40629cd52dcb4127b44306780ee0cc04e169929a5b43b70d40 Jul 07 19:22:30.245288 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.245264 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l6tpb" Jul 07 19:22:30.300645 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.300617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lks2m" event={"ID":"3266e501-d40c-4017-bbc7-bdf001361f1f","Type":"ContainerStarted","Data":"b868b9ef0ca1fa40629cd52dcb4127b44306780ee0cc04e169929a5b43b70d40"} Jul 07 19:22:30.301889 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.301857 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wwmc8" event={"ID":"9384fe94-f3cc-41ae-a897-53bc166f00b6","Type":"ContainerStarted","Data":"0f99383171277cf786bbb057fcdebfa5220396d60d6976c7f08ae6b018e1942e"} Jul 07 19:22:30.303481 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.303454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7fd877-nkhx8" event={"ID":"5ff2082c-f500-43b6-b11f-bab5f407170f","Type":"ContainerStarted","Data":"a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86"} Jul 07 19:22:30.303599 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.303505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7fd877-nkhx8" event={"ID":"5ff2082c-f500-43b6-b11f-bab5f407170f","Type":"ContainerStarted","Data":"b96c994b74049f260eb7e68be7cfa0b3838e6073784b38c7c8bdd26652914438"} Jul 07 19:22:30.304585 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.304560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gs7hm" event={"ID":"6be412fb-041b-44dc-b9a9-fbd8792bfd19","Type":"ContainerStarted","Data":"7cbe4aef7dd6fe1b7e603be875feec000af2b983ffd8995f1dfca7a33f9a9a60"} Jul 07 19:22:30.307515 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.307496 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"acf91af3976258a0f5eaef6b9a2ff9d948206d6f0bd92a31338fff385746e2de"} Jul 07 19:22:30.307610 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.307550 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"d0f324c8b25eca5507d3fb1a41cb814c4608ddf6f0d7aa5aee8c2d9487b181de"} Jul 07 19:22:30.307610 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.307583 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" event={"ID":"2cab8f4d-b087-434f-a4c5-41afd5161c63","Type":"ContainerStarted","Data":"421004b9b1404a3412edafda91d64edf00a5b87eead955814b4df1e54869fd64"} Jul 07 19:22:30.307702 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.307683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:30.324947 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.324897 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b4b7fd877-nkhx8" podStartSLOduration=2.324882067 podStartE2EDuration="2.324882067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:30.324779252 +0000 UTC m=+66.109764272" watchObservedRunningTime="2026-07-07 19:22:30.324882067 +0000 UTC m=+66.109867068" Jul 07 19:22:30.358645 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:30.358583 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" podStartSLOduration=1.544337623 podStartE2EDuration="5.358561912s" totalImagesPullingTime="3.814224289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:30.357665267 +0000 UTC m=+66.142650286" watchObservedRunningTime="2026-07-07 19:22:30.358561912 +0000 UTC m=+66.143546912" Jul 07 19:22:31.316822 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:31.315960 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" event={"ID":"286e1d46-4c13-4955-a6a7-bd623500eedd","Type":"ContainerStarted","Data":"e1e97df564f9b487a34442d0b850a3fc8248cbd188a1a18e7a1eaef0969449a1"} Jul 07 19:22:31.316822 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:31.316799 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:31.323169 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:31.323096 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" Jul 07 19:22:31.350242 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:31.350182 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qvqs9" podStartSLOduration=2.226145072 podStartE2EDuration="4.350160763s" totalImagesPullingTime="2.124015691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:31.348986788 +0000 UTC m=+67.133971784" watchObservedRunningTime="2026-07-07 19:22:31.350160763 +0000 UTC m=+67.135145766" Jul 07 19:22:32.322571 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:32.322527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wwmc8" event={"ID":"9384fe94-f3cc-41ae-a897-53bc166f00b6","Type":"ContainerStarted","Data":"689bd38dd528f3031fe66b6c204187c2261530e2788d2f27feee35915525ee8f"} Jul 07 19:22:32.323028 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:32.322595 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wwmc8" event={"ID":"9384fe94-f3cc-41ae-a897-53bc166f00b6","Type":"ContainerStarted","Data":"4444dd7633d7e608e0680d660bcf7020fb77940cb1f7c672649bae694cdb1534"} Jul 07 19:22:32.342310 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:32.342198 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wwmc8" podStartSLOduration=65.508529955 podStartE2EDuration="1m7.34217466s" totalImagesPullingTime="1.833644705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:32.341438675 +0000 UTC m=+68.126423739" watchObservedRunningTime="2026-07-07 19:22:32.34217466 +0000 UTC m=+68.127159661" Jul 07 19:22:33.988212 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:33.988178 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b4b7fd877-nkhx8"] Jul 07 19:22:34.021932 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.021897 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6849dc9fd6-8l4wm"] Jul 07 19:22:34.024656 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.024631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.037005 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.036963 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6849dc9fd6-8l4wm"] Jul 07 19:22:34.170261 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-oauth-config\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.170261 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-config\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.170532 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wzv4\" (UniqueName: \"kubernetes.io/projected/ad717994-b1ab-42c0-a4d2-ad4802b97405-kube-api-access-4wzv4\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.170532 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-oauth-serving-cert\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.170532 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-serving-cert\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.170532 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-service-ca\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.170532 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.170492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-trusted-ca-bundle\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271205 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-trusted-ca-bundle\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271205 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-oauth-config\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271432 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-config\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271432 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wzv4\" (UniqueName: \"kubernetes.io/projected/ad717994-b1ab-42c0-a4d2-ad4802b97405-kube-api-access-4wzv4\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-oauth-serving-cert\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-serving-cert\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.271539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.271529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-service-ca\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.272431 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.272400 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-trusted-ca-bundle\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.272557 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.272406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-config\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.272557 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.272534 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-service-ca\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.272699 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.272667 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-oauth-serving-cert\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.274359 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.274338 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-serving-cert\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.274453 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.274410 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-oauth-config\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.281752 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.281726 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wzv4\" (UniqueName: \"kubernetes.io/projected/ad717994-b1ab-42c0-a4d2-ad4802b97405-kube-api-access-4wzv4\") pod \"console-6849dc9fd6-8l4wm\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:34.335166 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:34.335121 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:35.537865 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:35.537834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6849dc9fd6-8l4wm"] Jul 07 19:22:35.541752 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:22:35.541722 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad717994_b1ab_42c0_a4d2_ad4802b97405.slice/crio-d8b5436b31dafa442f9709a5599cea50893e5ab8aa049836d3ee2e39aeb3daed WatchSource:0}: Error finding container d8b5436b31dafa442f9709a5599cea50893e5ab8aa049836d3ee2e39aeb3daed: Status 404 returned error can't find the container with id d8b5436b31dafa442f9709a5599cea50893e5ab8aa049836d3ee2e39aeb3daed Jul 07 19:22:36.322711 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.322685 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c495498f6-wc5kj" Jul 07 19:22:36.342688 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.342653 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lks2m" event={"ID":"3266e501-d40c-4017-bbc7-bdf001361f1f","Type":"ContainerStarted","Data":"63e8679c3741982fd564be9bcf67de17d69778cca9cd4f4f92140164a08fbc4e"} Jul 07 19:22:36.342859 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.342775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:22:36.344364 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.344271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gs7hm" event={"ID":"6be412fb-041b-44dc-b9a9-fbd8792bfd19","Type":"ContainerStarted","Data":"a8d6e7a7dd7e9e5ec6855dff3c8a6f73f4abf037cc3d35ea58e411851aabee13"} Jul 07 19:22:36.345791 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.345771 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849dc9fd6-8l4wm" event={"ID":"ad717994-b1ab-42c0-a4d2-ad4802b97405","Type":"ContainerStarted","Data":"b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8"} Jul 07 19:22:36.345878 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.345807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849dc9fd6-8l4wm" event={"ID":"ad717994-b1ab-42c0-a4d2-ad4802b97405","Type":"ContainerStarted","Data":"d8b5436b31dafa442f9709a5599cea50893e5ab8aa049836d3ee2e39aeb3daed"} Jul 07 19:22:36.367362 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.367310 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gs7hm" podStartSLOduration=65.965151545 podStartE2EDuration="1m11.367295009s" totalImagesPullingTime="5.402143464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:36.366276418 +0000 UTC m=+72.151261453" watchObservedRunningTime="2026-07-07 19:22:36.367295009 +0000 UTC m=+72.152280009" Jul 07 19:22:36.382947 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.382898 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lks2m" podStartSLOduration=66.206430702 podStartE2EDuration="1m11.3828787s" totalImagesPullingTime="5.176447998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:36.382476689 +0000 UTC m=+72.167461689" watchObservedRunningTime="2026-07-07 19:22:36.3828787 +0000 UTC m=+72.167863699" Jul 07 19:22:36.400943 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:36.400816 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6849dc9fd6-8l4wm" podStartSLOduration=2.4007961 podStartE2EDuration="2.4007961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:36.400406364 +0000 UTC m=+72.185391420" watchObservedRunningTime="2026-07-07 19:22:36.4007961 +0000 UTC m=+72.185781101" Jul 07 19:22:39.173927 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:39.173883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:44.335420 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:44.335367 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:44.335861 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:44.335444 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:44.340170 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:44.340149 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:44.381322 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:44.381296 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:22:44.435745 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:44.435708 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79747b9866-pjgbn"] Jul 07 19:22:50.332811 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:50.332690 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5fb9c47d5d-j6ls8" podUID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" containerName="console" containerID="cri-o://ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447" gracePeriod=15 Jul 07 19:22:51.118953 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.118920 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:51.308583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.308557 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-service-ca\") pod \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " Jul 07 19:22:51.308761 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.308607 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-config\") pod \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " Jul 07 19:22:51.308761 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.308628 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-oauth-serving-cert\") pod \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " Jul 07 19:22:51.308761 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.308664 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-oauth-config\") pod \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " Jul 07 19:22:51.308761 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.308696 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjlpl\" (UniqueName: \"kubernetes.io/projected/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-kube-api-access-zjlpl\") pod \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " Jul 07 19:22:51.308761 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.308732 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-serving-cert\") pod \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\" (UID: \"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e\") " Jul 07 19:22:51.309062 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.309031 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-config" (OuterVolumeSpecName: "console-config") pod "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" (UID: "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:51.309118 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.309056 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-service-ca" (OuterVolumeSpecName: "service-ca") pod "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" (UID: "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:51.309286 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.309067 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" (UID: "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:51.314355 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.314317 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" (UID: "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:22:51.314882 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.314856 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-kube-api-access-zjlpl" (OuterVolumeSpecName: "kube-api-access-zjlpl") pod "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" (UID: "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e"). InnerVolumeSpecName "kube-api-access-zjlpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:22:51.315061 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.315042 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" (UID: "f13e8cf2-a9ef-4eef-86d3-2f008a75e44e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:22:51.404546 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.404509 2580 generic.go:356] "Generic (PLEG): container finished" podID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" containerID="ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447" exitCode=0 Jul 07 19:22:51.404990 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.404574 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb9c47d5d-j6ls8" Jul 07 19:22:51.404990 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.404577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb9c47d5d-j6ls8" event={"ID":"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e","Type":"ContainerDied","Data":"ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447"} Jul 07 19:22:51.404990 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.404713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb9c47d5d-j6ls8" event={"ID":"f13e8cf2-a9ef-4eef-86d3-2f008a75e44e","Type":"ContainerDied","Data":"64f05107a0e6073722f901cc729dc4934695a2be40be52fc1c9cce353cf44af7"} Jul 07 19:22:51.404990 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.404741 2580 scope.go:117] "RemoveContainer" containerID="ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447" Jul 07 19:22:51.409247 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.409228 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-oauth-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:51.409247 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.409247 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjlpl\" (UniqueName: \"kubernetes.io/projected/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-kube-api-access-zjlpl\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:51.409388 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.409259 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:51.409388 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.409268 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-service-ca\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:51.409388 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.409276 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-console-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:51.409388 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.409283 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e-oauth-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:51.413815 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.413798 2580 scope.go:117] "RemoveContainer" containerID="ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447" Jul 07 19:22:51.414104 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:51.414085 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447\": container with ID starting with ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447 not found: ID does not exist" containerID="ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447" Jul 07 19:22:51.414178 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.414114 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447"} err="failed to get container status \"ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447\": rpc error: code = NotFound desc = could not find container \"ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447\": container with ID starting with ab6c68941d7849f0ccad5451272e517b3bef42464144edf298118fd965fec447 not found: ID does not exist" Jul 07 19:22:51.427964 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.427939 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb9c47d5d-j6ls8"] Jul 07 19:22:51.432848 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:51.432827 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fb9c47d5d-j6ls8"] Jul 07 19:22:52.917124 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:52.917091 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" path="/var/lib/kubelet/pods/f13e8cf2-a9ef-4eef-86d3-2f008a75e44e/volumes" Jul 07 19:22:59.009728 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.009662 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b4b7fd877-nkhx8" podUID="5ff2082c-f500-43b6-b11f-bab5f407170f" containerName="console" containerID="cri-o://a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86" gracePeriod=15 Jul 07 19:22:59.252583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.252560 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:59.267356 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267286 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-oauth-serving-cert\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267356 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267339 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-serving-cert\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267356 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267360 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-oauth-config\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267606 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267394 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-console-config\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267606 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267431 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-service-ca\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267606 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267463 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc8bz\" (UniqueName: \"kubernetes.io/projected/5ff2082c-f500-43b6-b11f-bab5f407170f-kube-api-access-gc8bz\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267606 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267591 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-trusted-ca-bundle\") pod \"5ff2082c-f500-43b6-b11f-bab5f407170f\" (UID: \"5ff2082c-f500-43b6-b11f-bab5f407170f\") " Jul 07 19:22:59.267807 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267781 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:59.267870 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267844 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-console-config" (OuterVolumeSpecName: "console-config") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:59.267870 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.267850 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-service-ca" (OuterVolumeSpecName: "service-ca") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:59.268185 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.268162 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:59.269697 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.269672 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:22:59.269794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.269727 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff2082c-f500-43b6-b11f-bab5f407170f-kube-api-access-gc8bz" (OuterVolumeSpecName: "kube-api-access-gc8bz") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "kube-api-access-gc8bz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:22:59.269794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.269733 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5ff2082c-f500-43b6-b11f-bab5f407170f" (UID: "5ff2082c-f500-43b6-b11f-bab5f407170f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:22:59.368137 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368092 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-console-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.368137 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368132 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-service-ca\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.368137 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368141 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc8bz\" (UniqueName: \"kubernetes.io/projected/5ff2082c-f500-43b6-b11f-bab5f407170f-kube-api-access-gc8bz\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.368137 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368150 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-trusted-ca-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.368402 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368159 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff2082c-f500-43b6-b11f-bab5f407170f-oauth-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.368402 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368169 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.368402 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.368177 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff2082c-f500-43b6-b11f-bab5f407170f-console-oauth-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:59.432648 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.432611 2580 generic.go:356] "Generic (PLEG): container finished" podID="5ff2082c-f500-43b6-b11f-bab5f407170f" containerID="a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86" exitCode=0 Jul 07 19:22:59.432813 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.432663 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7fd877-nkhx8" Jul 07 19:22:59.432813 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.432660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7fd877-nkhx8" event={"ID":"5ff2082c-f500-43b6-b11f-bab5f407170f","Type":"ContainerDied","Data":"a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86"} Jul 07 19:22:59.432813 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.432803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7fd877-nkhx8" event={"ID":"5ff2082c-f500-43b6-b11f-bab5f407170f","Type":"ContainerDied","Data":"b96c994b74049f260eb7e68be7cfa0b3838e6073784b38c7c8bdd26652914438"} Jul 07 19:22:59.432987 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.432840 2580 scope.go:117] "RemoveContainer" containerID="a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86" Jul 07 19:22:59.440930 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.440910 2580 scope.go:117] "RemoveContainer" containerID="a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86" Jul 07 19:22:59.441208 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:22:59.441192 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86\": container with ID starting with a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86 not found: ID does not exist" containerID="a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86" Jul 07 19:22:59.441254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.441217 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86"} err="failed to get container status \"a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86\": rpc error: code = NotFound desc = could not find container \"a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86\": container with ID starting with a488b2982a1fa6c4916dd0bfbabf9103faca3048335d349a13574a2083ce4b86 not found: ID does not exist" Jul 07 19:22:59.454011 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.453982 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b4b7fd877-nkhx8"] Jul 07 19:22:59.458530 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:22:59.458510 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b4b7fd877-nkhx8"] Jul 07 19:23:00.915274 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:00.915240 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff2082c-f500-43b6-b11f-bab5f407170f" path="/var/lib/kubelet/pods/5ff2082c-f500-43b6-b11f-bab5f407170f/volumes" Jul 07 19:23:07.352311 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:07.352282 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lks2m" Jul 07 19:23:09.456649 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.456606 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79747b9866-pjgbn" podUID="61f914ed-895d-44b9-a8fc-0b516137c0eb" containerName="console" containerID="cri-o://e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1" gracePeriod=15 Jul 07 19:23:09.722213 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.722187 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:23:09.743493 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743465 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-trusted-ca-bundle\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.743626 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743500 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-oauth-config\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.743626 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743530 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-service-ca\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.743626 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743554 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-config\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.743626 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743588 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxl4l\" (UniqueName: \"kubernetes.io/projected/61f914ed-895d-44b9-a8fc-0b516137c0eb-kube-api-access-dxl4l\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.743626 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743623 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-oauth-serving-cert\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.744407 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743685 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-serving-cert\") pod \"61f914ed-895d-44b9-a8fc-0b516137c0eb\" (UID: \"61f914ed-895d-44b9-a8fc-0b516137c0eb\") " Jul 07 19:23:09.744407 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.743917 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:23:09.744407 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.744250 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-trusted-ca-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:09.744407 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.744332 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:23:09.744407 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.744364 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-config" (OuterVolumeSpecName: "console-config") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:23:09.744797 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.744609 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:23:09.746681 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.746630 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f914ed-895d-44b9-a8fc-0b516137c0eb-kube-api-access-dxl4l" (OuterVolumeSpecName: "kube-api-access-dxl4l") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "kube-api-access-dxl4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:23:09.746796 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.746715 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:23:09.746861 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.746789 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "61f914ed-895d-44b9-a8fc-0b516137c0eb" (UID: "61f914ed-895d-44b9-a8fc-0b516137c0eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:23:09.844611 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.844573 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-oauth-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:09.844611 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.844603 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-service-ca\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:09.844611 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.844615 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:09.844611 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.844622 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxl4l\" (UniqueName: \"kubernetes.io/projected/61f914ed-895d-44b9-a8fc-0b516137c0eb-kube-api-access-dxl4l\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:09.844877 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.844631 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61f914ed-895d-44b9-a8fc-0b516137c0eb-oauth-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:09.844877 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:09.844640 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61f914ed-895d-44b9-a8fc-0b516137c0eb-console-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:23:10.479427 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.479394 2580 generic.go:356] "Generic (PLEG): container finished" podID="61f914ed-895d-44b9-a8fc-0b516137c0eb" containerID="e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1" exitCode=0 Jul 07 19:23:10.479843 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.479436 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79747b9866-pjgbn" Jul 07 19:23:10.479843 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.479451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79747b9866-pjgbn" event={"ID":"61f914ed-895d-44b9-a8fc-0b516137c0eb","Type":"ContainerDied","Data":"e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1"} Jul 07 19:23:10.479843 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.479492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79747b9866-pjgbn" event={"ID":"61f914ed-895d-44b9-a8fc-0b516137c0eb","Type":"ContainerDied","Data":"5d0347d06c72897fca387335f1484ceee6fc360d1062c1e248346b7209f0f311"} Jul 07 19:23:10.479843 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.479511 2580 scope.go:117] "RemoveContainer" containerID="e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1" Jul 07 19:23:10.488316 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.488276 2580 scope.go:117] "RemoveContainer" containerID="e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1" Jul 07 19:23:10.488572 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:23:10.488551 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1\": container with ID starting with e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1 not found: ID does not exist" containerID="e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1" Jul 07 19:23:10.488668 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.488579 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1"} err="failed to get container status \"e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1\": rpc error: code = NotFound desc = could not find container \"e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1\": container with ID starting with e0a41fde97c5082145f1b280676be1899f69760d2584cf716f4cc36c989cb7b1 not found: ID does not exist" Jul 07 19:23:10.501082 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.501056 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79747b9866-pjgbn"] Jul 07 19:23:10.510174 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.510152 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79747b9866-pjgbn"] Jul 07 19:23:10.915702 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:10.915672 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61f914ed-895d-44b9-a8fc-0b516137c0eb" path="/var/lib/kubelet/pods/61f914ed-895d-44b9-a8fc-0b516137c0eb/volumes" Jul 07 19:23:58.734860 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.734828 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d7995947-qbw2c"] Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735121 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735133 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735140 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61f914ed-895d-44b9-a8fc-0b516137c0eb" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735145 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="61f914ed-895d-44b9-a8fc-0b516137c0eb" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735151 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff2082c-f500-43b6-b11f-bab5f407170f" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735157 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff2082c-f500-43b6-b11f-bab5f407170f" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735228 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="61f914ed-895d-44b9-a8fc-0b516137c0eb" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735235 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff2082c-f500-43b6-b11f-bab5f407170f" containerName="console" Jul 07 19:23:58.735346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.735241 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13e8cf2-a9ef-4eef-86d3-2f008a75e44e" containerName="console" Jul 07 19:23:58.737197 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.737177 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.748842 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.748818 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d7995947-qbw2c"] Jul 07 19:23:58.798731 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-config\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.798731 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-service-ca\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.798935 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-trusted-ca-bundle\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.798935 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-oauth-serving-cert\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.798935 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-serving-cert\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.798935 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-oauth-config\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.799087 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.798966 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8j59\" (UniqueName: \"kubernetes.io/projected/56c6653c-28f6-4daf-ba2e-7de689b1cd78-kube-api-access-k8j59\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900286 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-service-ca\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900399 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-trusted-ca-bundle\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900399 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-oauth-serving-cert\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900399 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-serving-cert\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900399 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-oauth-config\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900549 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900428 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8j59\" (UniqueName: \"kubernetes.io/projected/56c6653c-28f6-4daf-ba2e-7de689b1cd78-kube-api-access-k8j59\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.900549 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.900534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-config\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.901032 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.901009 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-service-ca\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.901178 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.901098 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-oauth-serving-cert\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.901238 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.901184 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-trusted-ca-bundle\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.901238 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.901230 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-config\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.902853 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.902834 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-oauth-config\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.902954 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.902935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-serving-cert\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:58.910049 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:58.910019 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8j59\" (UniqueName: \"kubernetes.io/projected/56c6653c-28f6-4daf-ba2e-7de689b1cd78-kube-api-access-k8j59\") pod \"console-6d7995947-qbw2c\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:59.046059 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:59.045952 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:23:59.193096 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:59.193074 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d7995947-qbw2c"] Jul 07 19:23:59.195463 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:23:59.195438 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c6653c_28f6_4daf_ba2e_7de689b1cd78.slice/crio-b88a657779a55b66aa5f2470db41d782119358fb260ef56fa532fbbabd83bfbc WatchSource:0}: Error finding container b88a657779a55b66aa5f2470db41d782119358fb260ef56fa532fbbabd83bfbc: Status 404 returned error can't find the container with id b88a657779a55b66aa5f2470db41d782119358fb260ef56fa532fbbabd83bfbc Jul 07 19:23:59.651244 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:59.651208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7995947-qbw2c" event={"ID":"56c6653c-28f6-4daf-ba2e-7de689b1cd78","Type":"ContainerStarted","Data":"6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3"} Jul 07 19:23:59.651439 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:59.651268 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7995947-qbw2c" event={"ID":"56c6653c-28f6-4daf-ba2e-7de689b1cd78","Type":"ContainerStarted","Data":"b88a657779a55b66aa5f2470db41d782119358fb260ef56fa532fbbabd83bfbc"} Jul 07 19:23:59.672872 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:23:59.672816 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d7995947-qbw2c" podStartSLOduration=1.672799472 podStartE2EDuration="1.672799472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:23:59.671123233 +0000 UTC m=+155.456108270" watchObservedRunningTime="2026-07-07 19:23:59.672799472 +0000 UTC m=+155.457784471" Jul 07 19:24:09.046186 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:09.046142 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:24:09.046186 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:09.046205 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:24:09.051029 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:09.051005 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:24:09.691231 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:09.691205 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:24:09.740215 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:09.740182 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6849dc9fd6-8l4wm"] Jul 07 19:24:34.760319 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:34.760254 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6849dc9fd6-8l4wm" podUID="ad717994-b1ab-42c0-a4d2-ad4802b97405" containerName="console" containerID="cri-o://b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8" gracePeriod=15 Jul 07 19:24:35.003770 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.003744 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:24:35.069825 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069794 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-serving-cert\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069835 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-oauth-config\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069858 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-oauth-serving-cert\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069888 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-service-ca\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069912 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-config\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069933 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wzv4\" (UniqueName: \"kubernetes.io/projected/ad717994-b1ab-42c0-a4d2-ad4802b97405-kube-api-access-4wzv4\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070004 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.069956 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-trusted-ca-bundle\") pod \"ad717994-b1ab-42c0-a4d2-ad4802b97405\" (UID: \"ad717994-b1ab-42c0-a4d2-ad4802b97405\") " Jul 07 19:24:35.070433 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.070314 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:24:35.070586 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.070413 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-service-ca" (OuterVolumeSpecName: "service-ca") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:24:35.070586 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.070465 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-config" (OuterVolumeSpecName: "console-config") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:24:35.070586 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.070493 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:24:35.072167 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.072134 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:24:35.072254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.072165 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad717994-b1ab-42c0-a4d2-ad4802b97405-kube-api-access-4wzv4" (OuterVolumeSpecName: "kube-api-access-4wzv4") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "kube-api-access-4wzv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:24:35.072254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.072182 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ad717994-b1ab-42c0-a4d2-ad4802b97405" (UID: "ad717994-b1ab-42c0-a4d2-ad4802b97405"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:24:35.170684 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170650 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-oauth-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.170841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170698 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-oauth-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.170841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170709 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-service-ca\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.170841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170718 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.170841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170728 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wzv4\" (UniqueName: \"kubernetes.io/projected/ad717994-b1ab-42c0-a4d2-ad4802b97405-kube-api-access-4wzv4\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.170841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170736 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad717994-b1ab-42c0-a4d2-ad4802b97405-trusted-ca-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.170841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.170744 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad717994-b1ab-42c0-a4d2-ad4802b97405-console-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:35.777592 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.777559 2580 generic.go:356] "Generic (PLEG): container finished" podID="ad717994-b1ab-42c0-a4d2-ad4802b97405" containerID="b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8" exitCode=0 Jul 07 19:24:35.777999 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.777610 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6849dc9fd6-8l4wm" Jul 07 19:24:35.777999 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.777598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849dc9fd6-8l4wm" event={"ID":"ad717994-b1ab-42c0-a4d2-ad4802b97405","Type":"ContainerDied","Data":"b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8"} Jul 07 19:24:35.777999 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.777730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6849dc9fd6-8l4wm" event={"ID":"ad717994-b1ab-42c0-a4d2-ad4802b97405","Type":"ContainerDied","Data":"d8b5436b31dafa442f9709a5599cea50893e5ab8aa049836d3ee2e39aeb3daed"} Jul 07 19:24:35.777999 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.777756 2580 scope.go:117] "RemoveContainer" containerID="b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8" Jul 07 19:24:35.785960 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.785942 2580 scope.go:117] "RemoveContainer" containerID="b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8" Jul 07 19:24:35.786278 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:24:35.786260 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8\": container with ID starting with b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8 not found: ID does not exist" containerID="b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8" Jul 07 19:24:35.786334 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.786289 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8"} err="failed to get container status \"b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8\": rpc error: code = NotFound desc = could not find container \"b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8\": container with ID starting with b5c9f5fab62ff0860564b9006dd21f70c9ecdea9c44f07d0359a5973bb5c3df8 not found: ID does not exist" Jul 07 19:24:35.803075 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.802961 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6849dc9fd6-8l4wm"] Jul 07 19:24:35.806965 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:35.806943 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6849dc9fd6-8l4wm"] Jul 07 19:24:36.915137 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:36.915107 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad717994-b1ab-42c0-a4d2-ad4802b97405" path="/var/lib/kubelet/pods/ad717994-b1ab-42c0-a4d2-ad4802b97405/volumes" Jul 07 19:24:40.452967 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.452930 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt"] Jul 07 19:24:40.453458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.453311 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad717994-b1ab-42c0-a4d2-ad4802b97405" containerName="console" Jul 07 19:24:40.453458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.453328 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad717994-b1ab-42c0-a4d2-ad4802b97405" containerName="console" Jul 07 19:24:40.453458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.453395 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad717994-b1ab-42c0-a4d2-ad4802b97405" containerName="console" Jul 07 19:24:40.460130 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.460102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.462772 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.462749 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 19:24:40.462877 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.462779 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 19:24:40.463439 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.463419 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dr8rm\"" Jul 07 19:24:40.468138 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.468117 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt"] Jul 07 19:24:40.510870 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.510833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.510870 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.510883 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.511141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.510950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjmcc\" (UniqueName: \"kubernetes.io/projected/327cef53-55d1-46c6-8728-0f97cda86bd0-kube-api-access-gjmcc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.611938 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.611902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.611938 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.611943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.612141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.611991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjmcc\" (UniqueName: \"kubernetes.io/projected/327cef53-55d1-46c6-8728-0f97cda86bd0-kube-api-access-gjmcc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.612341 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.612324 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.612385 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.612349 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.621517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.621496 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjmcc\" (UniqueName: \"kubernetes.io/projected/327cef53-55d1-46c6-8728-0f97cda86bd0-kube-api-access-gjmcc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.769618 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.769532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:40.898679 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:40.898648 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt"] Jul 07 19:24:40.901553 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:24:40.901529 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327cef53_55d1_46c6_8728_0f97cda86bd0.slice/crio-677d99be9e1611823776c27951c7276d9aaca0c1a1069ff9a6faac7d65ae9fa6 WatchSource:0}: Error finding container 677d99be9e1611823776c27951c7276d9aaca0c1a1069ff9a6faac7d65ae9fa6: Status 404 returned error can't find the container with id 677d99be9e1611823776c27951c7276d9aaca0c1a1069ff9a6faac7d65ae9fa6 Jul 07 19:24:41.799547 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:41.799512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" event={"ID":"327cef53-55d1-46c6-8728-0f97cda86bd0","Type":"ContainerStarted","Data":"677d99be9e1611823776c27951c7276d9aaca0c1a1069ff9a6faac7d65ae9fa6"} Jul 07 19:24:46.820501 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:46.820460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" event={"ID":"327cef53-55d1-46c6-8728-0f97cda86bd0","Type":"ContainerStarted","Data":"b2df4df9c68b7b66b6e9f65feba9b654be26ca97b6309f4e95361f5caf12dda4"} Jul 07 19:24:47.825601 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:47.825566 2580 generic.go:356] "Generic (PLEG): container finished" podID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerID="b2df4df9c68b7b66b6e9f65feba9b654be26ca97b6309f4e95361f5caf12dda4" exitCode=0 Jul 07 19:24:47.826189 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:47.825607 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" event={"ID":"327cef53-55d1-46c6-8728-0f97cda86bd0","Type":"ContainerDied","Data":"b2df4df9c68b7b66b6e9f65feba9b654be26ca97b6309f4e95361f5caf12dda4"} Jul 07 19:24:50.839339 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:50.839303 2580 generic.go:356] "Generic (PLEG): container finished" podID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerID="cdfbe15c4b6711c69997a8c66a072b1122f76ce0468732695eefcec0f46afd57" exitCode=0 Jul 07 19:24:50.839754 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:50.839402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" event={"ID":"327cef53-55d1-46c6-8728-0f97cda86bd0","Type":"ContainerDied","Data":"cdfbe15c4b6711c69997a8c66a072b1122f76ce0468732695eefcec0f46afd57"} Jul 07 19:24:57.868578 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:57.868545 2580 generic.go:356] "Generic (PLEG): container finished" podID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerID="81c901c7073408772e112c902204bcb2ec27a2498a69ee21c1d8172d0ec51795" exitCode=0 Jul 07 19:24:57.868958 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:57.868630 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" event={"ID":"327cef53-55d1-46c6-8728-0f97cda86bd0","Type":"ContainerDied","Data":"81c901c7073408772e112c902204bcb2ec27a2498a69ee21c1d8172d0ec51795"} Jul 07 19:24:59.005252 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.005226 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:59.062265 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.062235 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-util\") pod \"327cef53-55d1-46c6-8728-0f97cda86bd0\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " Jul 07 19:24:59.062426 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.062292 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-bundle\") pod \"327cef53-55d1-46c6-8728-0f97cda86bd0\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " Jul 07 19:24:59.062426 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.062348 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjmcc\" (UniqueName: \"kubernetes.io/projected/327cef53-55d1-46c6-8728-0f97cda86bd0-kube-api-access-gjmcc\") pod \"327cef53-55d1-46c6-8728-0f97cda86bd0\" (UID: \"327cef53-55d1-46c6-8728-0f97cda86bd0\") " Jul 07 19:24:59.062937 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.062916 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-bundle" (OuterVolumeSpecName: "bundle") pod "327cef53-55d1-46c6-8728-0f97cda86bd0" (UID: "327cef53-55d1-46c6-8728-0f97cda86bd0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:24:59.064610 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.064594 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/327cef53-55d1-46c6-8728-0f97cda86bd0-kube-api-access-gjmcc" (OuterVolumeSpecName: "kube-api-access-gjmcc") pod "327cef53-55d1-46c6-8728-0f97cda86bd0" (UID: "327cef53-55d1-46c6-8728-0f97cda86bd0"). InnerVolumeSpecName "kube-api-access-gjmcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:24:59.067869 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.067843 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-util" (OuterVolumeSpecName: "util") pod "327cef53-55d1-46c6-8728-0f97cda86bd0" (UID: "327cef53-55d1-46c6-8728-0f97cda86bd0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:24:59.163050 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.162945 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:59.163050 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.162995 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjmcc\" (UniqueName: \"kubernetes.io/projected/327cef53-55d1-46c6-8728-0f97cda86bd0-kube-api-access-gjmcc\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:59.163050 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.163007 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/327cef53-55d1-46c6-8728-0f97cda86bd0-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:24:59.877533 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.877503 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" Jul 07 19:24:59.877703 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.877506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mhknt" event={"ID":"327cef53-55d1-46c6-8728-0f97cda86bd0","Type":"ContainerDied","Data":"677d99be9e1611823776c27951c7276d9aaca0c1a1069ff9a6faac7d65ae9fa6"} Jul 07 19:24:59.877703 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:24:59.877625 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="677d99be9e1611823776c27951c7276d9aaca0c1a1069ff9a6faac7d65ae9fa6" Jul 07 19:25:03.423166 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423133 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42"] Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423397 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="pull" Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423407 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="pull" Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423419 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="extract" Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423424 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="extract" Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423432 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="util" Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423437 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="util" Jul 07 19:25:03.423517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.423482 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="327cef53-55d1-46c6-8728-0f97cda86bd0" containerName="extract" Jul 07 19:25:03.467170 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.467136 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42"] Jul 07 19:25:03.467323 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.467251 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.469966 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.469943 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 19:25:03.470215 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.469944 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:25:03.470364 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.470349 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mlmql\"" Jul 07 19:25:03.593794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.593759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn29d\" (UniqueName: \"kubernetes.io/projected/7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60-kube-api-access-fn29d\") pod \"cert-manager-operator-controller-manager-55c444bf86-2qv42\" (UID: \"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.593964 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.593808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2qv42\" (UID: \"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.694671 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.694595 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2qv42\" (UID: \"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.694787 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.694670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn29d\" (UniqueName: \"kubernetes.io/projected/7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60-kube-api-access-fn29d\") pod \"cert-manager-operator-controller-manager-55c444bf86-2qv42\" (UID: \"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.695012 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.694993 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2qv42\" (UID: \"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.704209 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.704181 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn29d\" (UniqueName: \"kubernetes.io/projected/7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60-kube-api-access-fn29d\") pod \"cert-manager-operator-controller-manager-55c444bf86-2qv42\" (UID: \"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.776189 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.776152 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" Jul 07 19:25:03.921751 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:03.921719 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42"] Jul 07 19:25:03.925310 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:03.925280 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d91f7eb_0d8d_4e70_a17b_89d87f6a9c60.slice/crio-bc768c985e26511d43ed904a38de7705f0c300ada8e6f9a001c9dbdd9d9c597a WatchSource:0}: Error finding container bc768c985e26511d43ed904a38de7705f0c300ada8e6f9a001c9dbdd9d9c597a: Status 404 returned error can't find the container with id bc768c985e26511d43ed904a38de7705f0c300ada8e6f9a001c9dbdd9d9c597a Jul 07 19:25:04.896818 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:04.896782 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" event={"ID":"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60","Type":"ContainerStarted","Data":"bc768c985e26511d43ed904a38de7705f0c300ada8e6f9a001c9dbdd9d9c597a"} Jul 07 19:25:09.924872 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:09.924837 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" event={"ID":"7d91f7eb-0d8d-4e70-a17b-89d87f6a9c60","Type":"ContainerStarted","Data":"2ddd6ff36c8cf8c4b4a8655c20f7fc6f1cb99037075636a64fefefe36b0c7f53"} Jul 07 19:25:09.948641 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:09.948589 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2qv42" podStartSLOduration=2.012316398 podStartE2EDuration="6.948574725s" totalImagesPullingTime="4.936258327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:09.94743665 +0000 UTC m=+225.732421668" watchObservedRunningTime="2026-07-07 19:25:09.948574725 +0000 UTC m=+225.733559725" Jul 07 19:25:10.920964 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:10.920929 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm"] Jul 07 19:25:10.923689 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:10.923673 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:10.926086 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:10.926058 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 19:25:10.926449 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:10.926096 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dr8rm\"" Jul 07 19:25:10.926903 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:10.926877 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 19:25:10.932793 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:10.932772 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm"] Jul 07 19:25:11.055017 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.054963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.055191 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.055025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.055191 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.055065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl25l\" (UniqueName: \"kubernetes.io/projected/09d072b2-6835-4e53-867c-c94ae3a5d516-kube-api-access-jl25l\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.086610 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.086577 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v8wln"] Jul 07 19:25:11.089006 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.088990 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.092236 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.092216 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 19:25:11.092381 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.092303 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 19:25:11.092381 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.092300 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-w4s6g\"" Jul 07 19:25:11.100915 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.100893 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v8wln"] Jul 07 19:25:11.155861 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.155820 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a596a5b-4df0-4d27-a254-86a59e5cabdf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v8wln\" (UID: \"3a596a5b-4df0-4d27-a254-86a59e5cabdf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.156060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.155881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.156060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.155943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.156060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.156015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl25l\" (UniqueName: \"kubernetes.io/projected/09d072b2-6835-4e53-867c-c94ae3a5d516-kube-api-access-jl25l\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.156219 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.156059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77swc\" (UniqueName: \"kubernetes.io/projected/3a596a5b-4df0-4d27-a254-86a59e5cabdf-kube-api-access-77swc\") pod \"cert-manager-webhook-6486ffd997-v8wln\" (UID: \"3a596a5b-4df0-4d27-a254-86a59e5cabdf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.156275 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.156220 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.156275 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.156256 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.168588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.168558 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl25l\" (UniqueName: \"kubernetes.io/projected/09d072b2-6835-4e53-867c-c94ae3a5d516-kube-api-access-jl25l\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.234104 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.234018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:11.257159 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.257133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77swc\" (UniqueName: \"kubernetes.io/projected/3a596a5b-4df0-4d27-a254-86a59e5cabdf-kube-api-access-77swc\") pod \"cert-manager-webhook-6486ffd997-v8wln\" (UID: \"3a596a5b-4df0-4d27-a254-86a59e5cabdf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.257320 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.257190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a596a5b-4df0-4d27-a254-86a59e5cabdf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v8wln\" (UID: \"3a596a5b-4df0-4d27-a254-86a59e5cabdf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.268087 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.268055 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a596a5b-4df0-4d27-a254-86a59e5cabdf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v8wln\" (UID: \"3a596a5b-4df0-4d27-a254-86a59e5cabdf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.268233 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.268147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77swc\" (UniqueName: \"kubernetes.io/projected/3a596a5b-4df0-4d27-a254-86a59e5cabdf-kube-api-access-77swc\") pod \"cert-manager-webhook-6486ffd997-v8wln\" (UID: \"3a596a5b-4df0-4d27-a254-86a59e5cabdf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.385146 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.385116 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm"] Jul 07 19:25:11.388194 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:11.388169 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d072b2_6835_4e53_867c_c94ae3a5d516.slice/crio-5553ed50508b974e6e193b1a4cbe853671a9c5be40f742550137961da132cff5 WatchSource:0}: Error finding container 5553ed50508b974e6e193b1a4cbe853671a9c5be40f742550137961da132cff5: Status 404 returned error can't find the container with id 5553ed50508b974e6e193b1a4cbe853671a9c5be40f742550137961da132cff5 Jul 07 19:25:11.409147 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.409127 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:11.548055 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.548028 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v8wln"] Jul 07 19:25:11.550674 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:11.550649 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a596a5b_4df0_4d27_a254_86a59e5cabdf.slice/crio-0f34e72fbae843eef2345432e57066eb4d3c47726a15f28b1c7dac10856ad8af WatchSource:0}: Error finding container 0f34e72fbae843eef2345432e57066eb4d3c47726a15f28b1c7dac10856ad8af: Status 404 returned error can't find the container with id 0f34e72fbae843eef2345432e57066eb4d3c47726a15f28b1c7dac10856ad8af Jul 07 19:25:11.934199 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.934154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" event={"ID":"3a596a5b-4df0-4d27-a254-86a59e5cabdf","Type":"ContainerStarted","Data":"0f34e72fbae843eef2345432e57066eb4d3c47726a15f28b1c7dac10856ad8af"} Jul 07 19:25:11.935618 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.935595 2580 generic.go:356] "Generic (PLEG): container finished" podID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerID="c3614f63839930f7aee28b4b0f8a3124416d1bb1e086651ee97aba17be96a5ab" exitCode=0 Jul 07 19:25:11.935775 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.935647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" event={"ID":"09d072b2-6835-4e53-867c-c94ae3a5d516","Type":"ContainerDied","Data":"c3614f63839930f7aee28b4b0f8a3124416d1bb1e086651ee97aba17be96a5ab"} Jul 07 19:25:11.935775 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:11.935707 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" event={"ID":"09d072b2-6835-4e53-867c-c94ae3a5d516","Type":"ContainerStarted","Data":"5553ed50508b974e6e193b1a4cbe853671a9c5be40f742550137961da132cff5"} Jul 07 19:25:12.568692 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.568620 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb"] Jul 07 19:25:12.571231 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.571206 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.574256 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.574232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fc6qg\"" Jul 07 19:25:12.582324 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.582295 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb"] Jul 07 19:25:12.669959 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.669923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/069c7246-298e-4d62-88a6-33b883f31a4a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pfkwb\" (UID: \"069c7246-298e-4d62-88a6-33b883f31a4a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.670134 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.669993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlz6l\" (UniqueName: \"kubernetes.io/projected/069c7246-298e-4d62-88a6-33b883f31a4a-kube-api-access-wlz6l\") pod \"cert-manager-cainjector-559cdd8c87-pfkwb\" (UID: \"069c7246-298e-4d62-88a6-33b883f31a4a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.771272 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.771129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/069c7246-298e-4d62-88a6-33b883f31a4a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pfkwb\" (UID: \"069c7246-298e-4d62-88a6-33b883f31a4a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.771272 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.771195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlz6l\" (UniqueName: \"kubernetes.io/projected/069c7246-298e-4d62-88a6-33b883f31a4a-kube-api-access-wlz6l\") pod \"cert-manager-cainjector-559cdd8c87-pfkwb\" (UID: \"069c7246-298e-4d62-88a6-33b883f31a4a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.785006 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.782923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/069c7246-298e-4d62-88a6-33b883f31a4a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pfkwb\" (UID: \"069c7246-298e-4d62-88a6-33b883f31a4a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.785006 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.783522 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlz6l\" (UniqueName: \"kubernetes.io/projected/069c7246-298e-4d62-88a6-33b883f31a4a-kube-api-access-wlz6l\") pod \"cert-manager-cainjector-559cdd8c87-pfkwb\" (UID: \"069c7246-298e-4d62-88a6-33b883f31a4a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:12.884533 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:12.884420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" Jul 07 19:25:14.671100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.671078 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb"] Jul 07 19:25:14.674406 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:14.674370 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod069c7246_298e_4d62_88a6_33b883f31a4a.slice/crio-794c5a2fc7e1d9bf46311ee825d06993492849306f5ede4c3fc267bcd15c9607 WatchSource:0}: Error finding container 794c5a2fc7e1d9bf46311ee825d06993492849306f5ede4c3fc267bcd15c9607: Status 404 returned error can't find the container with id 794c5a2fc7e1d9bf46311ee825d06993492849306f5ede4c3fc267bcd15c9607 Jul 07 19:25:14.950170 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.950138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" event={"ID":"3a596a5b-4df0-4d27-a254-86a59e5cabdf","Type":"ContainerStarted","Data":"6d25b21170a68d30392607a205491e8086e4ba6ce1ce5e6740368137573f062a"} Jul 07 19:25:14.950379 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.950187 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:14.952211 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.952186 2580 generic.go:356] "Generic (PLEG): container finished" podID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerID="18b57764ec4116355192a79be98de9df2f1098e0f0dfe923db8a3978db84755b" exitCode=0 Jul 07 19:25:14.952326 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.952251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" event={"ID":"09d072b2-6835-4e53-867c-c94ae3a5d516","Type":"ContainerDied","Data":"18b57764ec4116355192a79be98de9df2f1098e0f0dfe923db8a3978db84755b"} Jul 07 19:25:14.953870 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.953850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" event={"ID":"069c7246-298e-4d62-88a6-33b883f31a4a","Type":"ContainerStarted","Data":"53364034c0f275c5050664f1aab60d85c67fae3c4afabb857fcbfd91c50c209c"} Jul 07 19:25:14.953984 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.953904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" event={"ID":"069c7246-298e-4d62-88a6-33b883f31a4a","Type":"ContainerStarted","Data":"794c5a2fc7e1d9bf46311ee825d06993492849306f5ede4c3fc267bcd15c9607"} Jul 07 19:25:14.979155 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:14.979103 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" podStartSLOduration=0.982090929 podStartE2EDuration="3.979081644s" totalImagesPullingTime="2.996990715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:14.977539704 +0000 UTC m=+230.762524723" watchObservedRunningTime="2026-07-07 19:25:14.979081644 +0000 UTC m=+230.764066646" Jul 07 19:25:15.016240 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:15.016189 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pfkwb" podStartSLOduration=3.016172351 podStartE2EDuration="3.016172351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:15.014743147 +0000 UTC m=+230.799728170" watchObservedRunningTime="2026-07-07 19:25:15.016172351 +0000 UTC m=+230.801157350" Jul 07 19:25:15.959714 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:15.959679 2580 generic.go:356] "Generic (PLEG): container finished" podID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerID="a5d8602332fc4a84a8bf5c6ca4a1d26f6f7285fd847695e69c5878a0dd83963b" exitCode=0 Jul 07 19:25:15.960167 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:15.959772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" event={"ID":"09d072b2-6835-4e53-867c-c94ae3a5d516","Type":"ContainerDied","Data":"a5d8602332fc4a84a8bf5c6ca4a1d26f6f7285fd847695e69c5878a0dd83963b"} Jul 07 19:25:17.084882 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.084856 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:17.207256 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.207224 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl25l\" (UniqueName: \"kubernetes.io/projected/09d072b2-6835-4e53-867c-c94ae3a5d516-kube-api-access-jl25l\") pod \"09d072b2-6835-4e53-867c-c94ae3a5d516\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " Jul 07 19:25:17.207256 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.207265 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-util\") pod \"09d072b2-6835-4e53-867c-c94ae3a5d516\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " Jul 07 19:25:17.207511 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.207314 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-bundle\") pod \"09d072b2-6835-4e53-867c-c94ae3a5d516\" (UID: \"09d072b2-6835-4e53-867c-c94ae3a5d516\") " Jul 07 19:25:17.207727 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.207702 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-bundle" (OuterVolumeSpecName: "bundle") pod "09d072b2-6835-4e53-867c-c94ae3a5d516" (UID: "09d072b2-6835-4e53-867c-c94ae3a5d516"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:25:17.209346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.209321 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09d072b2-6835-4e53-867c-c94ae3a5d516-kube-api-access-jl25l" (OuterVolumeSpecName: "kube-api-access-jl25l") pod "09d072b2-6835-4e53-867c-c94ae3a5d516" (UID: "09d072b2-6835-4e53-867c-c94ae3a5d516"). InnerVolumeSpecName "kube-api-access-jl25l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:25:17.211417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.211391 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-util" (OuterVolumeSpecName: "util") pod "09d072b2-6835-4e53-867c-c94ae3a5d516" (UID: "09d072b2-6835-4e53-867c-c94ae3a5d516"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:25:17.308413 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.308380 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl25l\" (UniqueName: \"kubernetes.io/projected/09d072b2-6835-4e53-867c-c94ae3a5d516-kube-api-access-jl25l\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:25:17.308413 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.308408 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:25:17.308413 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.308417 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09d072b2-6835-4e53-867c-c94ae3a5d516-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:25:17.969995 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.969947 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" event={"ID":"09d072b2-6835-4e53-867c-c94ae3a5d516","Type":"ContainerDied","Data":"5553ed50508b974e6e193b1a4cbe853671a9c5be40f742550137961da132cff5"} Jul 07 19:25:17.970166 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.970018 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5553ed50508b974e6e193b1a4cbe853671a9c5be40f742550137961da132cff5" Jul 07 19:25:17.970166 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:17.969963 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxmzrm" Jul 07 19:25:20.961999 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:20.961952 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-v8wln" Jul 07 19:25:24.193265 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193227 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk"] Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193598 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="extract" Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193614 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="extract" Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193629 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="pull" Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193636 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="pull" Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193649 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="util" Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193657 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="util" Jul 07 19:25:24.193739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.193720 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09d072b2-6835-4e53-867c-c94ae3a5d516" containerName="extract" Jul 07 19:25:24.195794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.195773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.198335 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.198302 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lctj2\"" Jul 07 19:25:24.198458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.198342 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:25:24.199070 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.199055 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 19:25:24.208447 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.208427 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk"] Jul 07 19:25:24.363411 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.363375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/935503c4-91a9-48e9-b937-da41de0b40f8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zkcgk\" (UID: \"935503c4-91a9-48e9-b937-da41de0b40f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.363574 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.363422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd9xj\" (UniqueName: \"kubernetes.io/projected/935503c4-91a9-48e9-b937-da41de0b40f8-kube-api-access-kd9xj\") pod \"openshift-lws-operator-fd8ccff4c-zkcgk\" (UID: \"935503c4-91a9-48e9-b937-da41de0b40f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.464716 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.464629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/935503c4-91a9-48e9-b937-da41de0b40f8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zkcgk\" (UID: \"935503c4-91a9-48e9-b937-da41de0b40f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.464716 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.464673 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd9xj\" (UniqueName: \"kubernetes.io/projected/935503c4-91a9-48e9-b937-da41de0b40f8-kube-api-access-kd9xj\") pod \"openshift-lws-operator-fd8ccff4c-zkcgk\" (UID: \"935503c4-91a9-48e9-b937-da41de0b40f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.465055 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.465036 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/935503c4-91a9-48e9-b937-da41de0b40f8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zkcgk\" (UID: \"935503c4-91a9-48e9-b937-da41de0b40f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.473998 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.473952 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd9xj\" (UniqueName: \"kubernetes.io/projected/935503c4-91a9-48e9-b937-da41de0b40f8-kube-api-access-kd9xj\") pod \"openshift-lws-operator-fd8ccff4c-zkcgk\" (UID: \"935503c4-91a9-48e9-b937-da41de0b40f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.505046 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.505008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" Jul 07 19:25:24.637002 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.636906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk"] Jul 07 19:25:24.639625 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:24.639594 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod935503c4_91a9_48e9_b937_da41de0b40f8.slice/crio-788d08571057a542d8945b23a0ce569a40db42d0f36dcd4fa0eae2717b9a6855 WatchSource:0}: Error finding container 788d08571057a542d8945b23a0ce569a40db42d0f36dcd4fa0eae2717b9a6855: Status 404 returned error can't find the container with id 788d08571057a542d8945b23a0ce569a40db42d0f36dcd4fa0eae2717b9a6855 Jul 07 19:25:24.999555 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:24.999518 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" event={"ID":"935503c4-91a9-48e9-b937-da41de0b40f8","Type":"ContainerStarted","Data":"788d08571057a542d8945b23a0ce569a40db42d0f36dcd4fa0eae2717b9a6855"} Jul 07 19:25:28.018154 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:28.018124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" event={"ID":"935503c4-91a9-48e9-b937-da41de0b40f8","Type":"ContainerStarted","Data":"30b90e83587a72b66edeee33149b1a473ece97a070cb832d82bf81340b7b3aef"} Jul 07 19:25:28.036037 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:28.035957 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkcgk" podStartSLOduration=0.882243513 podStartE2EDuration="4.035940539s" totalImagesPullingTime="3.153697026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:28.035143146 +0000 UTC m=+243.820128147" watchObservedRunningTime="2026-07-07 19:25:28.035940539 +0000 UTC m=+243.820925538" Jul 07 19:25:30.388306 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.388274 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp"] Jul 07 19:25:30.390683 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.390668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.393531 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.393511 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 19:25:30.393959 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.393933 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 19:25:30.394078 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.393990 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dr8rm\"" Jul 07 19:25:30.400060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.400040 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp"] Jul 07 19:25:30.518943 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.518906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.519140 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.518999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.519140 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.519047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vfdw\" (UniqueName: \"kubernetes.io/projected/141ac9e4-ce93-49ce-a147-a678231b34bc-kube-api-access-9vfdw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.619891 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.619851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.620124 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.619912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vfdw\" (UniqueName: \"kubernetes.io/projected/141ac9e4-ce93-49ce-a147-a678231b34bc-kube-api-access-9vfdw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.620124 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.619963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.620355 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.620326 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.620488 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.620337 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.621790 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.621769 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6wzks"] Jul 07 19:25:30.624178 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.624163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.626420 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.626404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2pfkv\"" Jul 07 19:25:30.629719 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.629697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vfdw\" (UniqueName: \"kubernetes.io/projected/141ac9e4-ce93-49ce-a147-a678231b34bc-kube-api-access-9vfdw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.637478 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.637457 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6wzks"] Jul 07 19:25:30.699728 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.699645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:30.721162 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.721132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxk5b\" (UniqueName: \"kubernetes.io/projected/36491b8b-32cd-40a4-9d5e-6286fb42931c-kube-api-access-mxk5b\") pod \"cert-manager-55b4d954d5-6wzks\" (UID: \"36491b8b-32cd-40a4-9d5e-6286fb42931c\") " pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.721359 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.721218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36491b8b-32cd-40a4-9d5e-6286fb42931c-bound-sa-token\") pod \"cert-manager-55b4d954d5-6wzks\" (UID: \"36491b8b-32cd-40a4-9d5e-6286fb42931c\") " pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.821680 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.821651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36491b8b-32cd-40a4-9d5e-6286fb42931c-bound-sa-token\") pod \"cert-manager-55b4d954d5-6wzks\" (UID: \"36491b8b-32cd-40a4-9d5e-6286fb42931c\") " pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.821868 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.821717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxk5b\" (UniqueName: \"kubernetes.io/projected/36491b8b-32cd-40a4-9d5e-6286fb42931c-kube-api-access-mxk5b\") pod \"cert-manager-55b4d954d5-6wzks\" (UID: \"36491b8b-32cd-40a4-9d5e-6286fb42931c\") " pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.830053 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.830028 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36491b8b-32cd-40a4-9d5e-6286fb42931c-bound-sa-token\") pod \"cert-manager-55b4d954d5-6wzks\" (UID: \"36491b8b-32cd-40a4-9d5e-6286fb42931c\") " pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.830298 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.830250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxk5b\" (UniqueName: \"kubernetes.io/projected/36491b8b-32cd-40a4-9d5e-6286fb42931c-kube-api-access-mxk5b\") pod \"cert-manager-55b4d954d5-6wzks\" (UID: \"36491b8b-32cd-40a4-9d5e-6286fb42931c\") " pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:30.846838 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.846804 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp"] Jul 07 19:25:30.850468 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:30.850382 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141ac9e4_ce93_49ce_a147_a678231b34bc.slice/crio-84d4a216106e424a7d6f78f1213b24f460410c946a2f5e9e05628866e3e5caee WatchSource:0}: Error finding container 84d4a216106e424a7d6f78f1213b24f460410c946a2f5e9e05628866e3e5caee: Status 404 returned error can't find the container with id 84d4a216106e424a7d6f78f1213b24f460410c946a2f5e9e05628866e3e5caee Jul 07 19:25:30.939314 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:30.939282 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6wzks" Jul 07 19:25:31.032311 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:31.032273 2580 generic.go:356] "Generic (PLEG): container finished" podID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerID="671f19fbb33278cf4ba0ec8d7cd5199bd4d9d2dbd6faeb53e5e2546516d6c2f4" exitCode=0 Jul 07 19:25:31.032463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:31.032323 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" event={"ID":"141ac9e4-ce93-49ce-a147-a678231b34bc","Type":"ContainerDied","Data":"671f19fbb33278cf4ba0ec8d7cd5199bd4d9d2dbd6faeb53e5e2546516d6c2f4"} Jul 07 19:25:31.032463 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:31.032374 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" event={"ID":"141ac9e4-ce93-49ce-a147-a678231b34bc","Type":"ContainerStarted","Data":"84d4a216106e424a7d6f78f1213b24f460410c946a2f5e9e05628866e3e5caee"} Jul 07 19:25:31.081251 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:31.081214 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6wzks"] Jul 07 19:25:31.086704 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:31.086679 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36491b8b_32cd_40a4_9d5e_6286fb42931c.slice/crio-8cafee9a995ece03e68c6a004ca7ada8630fd356b56b4f4ed6e571441186bbff WatchSource:0}: Error finding container 8cafee9a995ece03e68c6a004ca7ada8630fd356b56b4f4ed6e571441186bbff: Status 404 returned error can't find the container with id 8cafee9a995ece03e68c6a004ca7ada8630fd356b56b4f4ed6e571441186bbff Jul 07 19:25:32.038431 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:32.038331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6wzks" event={"ID":"36491b8b-32cd-40a4-9d5e-6286fb42931c","Type":"ContainerStarted","Data":"0386b8175215b51b27f4d96906bba3d7002b40f39d0223544579ac2e76d04a70"} Jul 07 19:25:32.038431 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:32.038376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6wzks" event={"ID":"36491b8b-32cd-40a4-9d5e-6286fb42931c","Type":"ContainerStarted","Data":"8cafee9a995ece03e68c6a004ca7ada8630fd356b56b4f4ed6e571441186bbff"} Jul 07 19:25:32.040086 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:32.040060 2580 generic.go:356] "Generic (PLEG): container finished" podID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerID="4f18bf4eca5e0d436536eb960ed149b31c1d930678cd3d0c303585aee4eb0d39" exitCode=0 Jul 07 19:25:32.040207 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:32.040113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" event={"ID":"141ac9e4-ce93-49ce-a147-a678231b34bc","Type":"ContainerDied","Data":"4f18bf4eca5e0d436536eb960ed149b31c1d930678cd3d0c303585aee4eb0d39"} Jul 07 19:25:32.062266 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:32.062219 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6wzks" podStartSLOduration=2.062204499 podStartE2EDuration="2.062204499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:32.060796783 +0000 UTC m=+247.845781797" watchObservedRunningTime="2026-07-07 19:25:32.062204499 +0000 UTC m=+247.847189499" Jul 07 19:25:33.046344 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:33.046307 2580 generic.go:356] "Generic (PLEG): container finished" podID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerID="c442a0f81b9f5024a278490f3f5ba4f2af9bf2a607317336e1ac6e2c342a02ae" exitCode=0 Jul 07 19:25:33.046764 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:33.046394 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" event={"ID":"141ac9e4-ce93-49ce-a147-a678231b34bc","Type":"ContainerDied","Data":"c442a0f81b9f5024a278490f3f5ba4f2af9bf2a607317336e1ac6e2c342a02ae"} Jul 07 19:25:34.176389 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.176367 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:34.248470 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.248413 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-util\") pod \"141ac9e4-ce93-49ce-a147-a678231b34bc\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " Jul 07 19:25:34.248666 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.248519 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vfdw\" (UniqueName: \"kubernetes.io/projected/141ac9e4-ce93-49ce-a147-a678231b34bc-kube-api-access-9vfdw\") pod \"141ac9e4-ce93-49ce-a147-a678231b34bc\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " Jul 07 19:25:34.248666 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.248619 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-bundle\") pod \"141ac9e4-ce93-49ce-a147-a678231b34bc\" (UID: \"141ac9e4-ce93-49ce-a147-a678231b34bc\") " Jul 07 19:25:34.249541 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.249499 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-bundle" (OuterVolumeSpecName: "bundle") pod "141ac9e4-ce93-49ce-a147-a678231b34bc" (UID: "141ac9e4-ce93-49ce-a147-a678231b34bc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:25:34.250837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.250809 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141ac9e4-ce93-49ce-a147-a678231b34bc-kube-api-access-9vfdw" (OuterVolumeSpecName: "kube-api-access-9vfdw") pod "141ac9e4-ce93-49ce-a147-a678231b34bc" (UID: "141ac9e4-ce93-49ce-a147-a678231b34bc"). InnerVolumeSpecName "kube-api-access-9vfdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:25:34.254539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.254503 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-util" (OuterVolumeSpecName: "util") pod "141ac9e4-ce93-49ce-a147-a678231b34bc" (UID: "141ac9e4-ce93-49ce-a147-a678231b34bc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:25:34.349305 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.349221 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:25:34.349305 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.349251 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vfdw\" (UniqueName: \"kubernetes.io/projected/141ac9e4-ce93-49ce-a147-a678231b34bc-kube-api-access-9vfdw\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:25:34.349305 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:34.349263 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141ac9e4-ce93-49ce-a147-a678231b34bc-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:25:35.056233 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:35.056194 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" event={"ID":"141ac9e4-ce93-49ce-a147-a678231b34bc","Type":"ContainerDied","Data":"84d4a216106e424a7d6f78f1213b24f460410c946a2f5e9e05628866e3e5caee"} Jul 07 19:25:35.056233 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:35.056230 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdv2lp" Jul 07 19:25:35.056427 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:35.056252 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84d4a216106e424a7d6f78f1213b24f460410c946a2f5e9e05628866e3e5caee" Jul 07 19:25:47.788350 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788264 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh"] Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788585 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="util" Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788598 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="util" Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788611 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="pull" Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788618 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="pull" Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788623 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="extract" Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788630 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="extract" Jul 07 19:25:47.788715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.788685 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="141ac9e4-ce93-49ce-a147-a678231b34bc" containerName="extract" Jul 07 19:25:47.790877 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.790856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.802381 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.802364 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 19:25:47.802549 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.802533 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 19:25:47.802613 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.802601 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-smps6\"" Jul 07 19:25:47.803565 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.803550 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 19:25:47.804129 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.804117 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 19:25:47.835616 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.835583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-tmp\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.835616 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.835625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n8g2\" (UniqueName: \"kubernetes.io/projected/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-kube-api-access-2n8g2\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.835868 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.835687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.835868 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.835790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-manifests\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.835868 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.835849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.837217 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.837192 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh"] Jul 07 19:25:47.937183 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.937151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-tmp\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.937183 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.937187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n8g2\" (UniqueName: \"kubernetes.io/projected/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-kube-api-access-2n8g2\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.937423 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.937229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.937423 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.937266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-manifests\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.937423 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.937284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.937705 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.937681 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-manifests\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.939430 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.939397 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-tmp\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.939655 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.939636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.939701 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.939673 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:47.955708 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:47.955684 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n8g2\" (UniqueName: \"kubernetes.io/projected/1f7143ad-8fbc-4179-a29e-de11b7d40c1b-kube-api-access-2n8g2\") pod \"opendatahub-operator-controller-manager-79b594cbdb-jtqvh\" (UID: \"1f7143ad-8fbc-4179-a29e-de11b7d40c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:48.100161 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:48.100118 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:48.233380 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:48.233351 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh"] Jul 07 19:25:48.235715 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:25:48.235689 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7143ad_8fbc_4179_a29e_de11b7d40c1b.slice/crio-a2cafffa2844b8bd733681f0bf33ef2302b556b107b41041eddfafc819aa1058 WatchSource:0}: Error finding container a2cafffa2844b8bd733681f0bf33ef2302b556b107b41041eddfafc819aa1058: Status 404 returned error can't find the container with id a2cafffa2844b8bd733681f0bf33ef2302b556b107b41041eddfafc819aa1058 Jul 07 19:25:49.119117 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:49.119080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" event={"ID":"1f7143ad-8fbc-4179-a29e-de11b7d40c1b","Type":"ContainerStarted","Data":"a2cafffa2844b8bd733681f0bf33ef2302b556b107b41041eddfafc819aa1058"} Jul 07 19:25:52.138357 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:52.138321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" event={"ID":"1f7143ad-8fbc-4179-a29e-de11b7d40c1b","Type":"ContainerStarted","Data":"0d798583d12fb0de616df959eff2eccdce4b022099df8ca781ec2031dd69f504"} Jul 07 19:25:53.144479 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:53.144445 2580 generic.go:356] "Generic (PLEG): container finished" podID="1f7143ad-8fbc-4179-a29e-de11b7d40c1b" containerID="0d798583d12fb0de616df959eff2eccdce4b022099df8ca781ec2031dd69f504" exitCode=0 Jul 07 19:25:53.144868 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:53.144492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" event={"ID":"1f7143ad-8fbc-4179-a29e-de11b7d40c1b","Type":"ContainerDied","Data":"0d798583d12fb0de616df959eff2eccdce4b022099df8ca781ec2031dd69f504"} Jul 07 19:25:54.151037 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:54.151005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" event={"ID":"1f7143ad-8fbc-4179-a29e-de11b7d40c1b","Type":"ContainerStarted","Data":"c7c81627488d7bf02de08b68af1d84b9f3f7715d623ed3b8eea444ebf31dfbbc"} Jul 07 19:25:54.151408 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:54.151134 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:25:54.176212 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:25:54.176158 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" podStartSLOduration=2.486738514 podStartE2EDuration="7.176134376s" totalImagesPullingTime="3.558827943s" totalInitContainerRuntime="1.130567919s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:54.17486336 +0000 UTC m=+269.959848397" watchObservedRunningTime="2026-07-07 19:25:54.176134376 +0000 UTC m=+269.961119377" Jul 07 19:26:05.158466 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:05.158434 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-jtqvh" Jul 07 19:26:07.722490 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.722455 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-w2cqz"] Jul 07 19:26:07.729070 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.729050 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:07.733616 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.733594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-p47ck\"" Jul 07 19:26:07.735065 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.735042 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 19:26:07.736695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.736671 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-w2cqz"] Jul 07 19:26:07.798245 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.798203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed0984e8-4daf-49c8-b7d9-aff614349488-cert\") pod \"odh-model-controller-7998c67dc4-w2cqz\" (UID: \"ed0984e8-4daf-49c8-b7d9-aff614349488\") " pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:07.798433 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.798266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cltk\" (UniqueName: \"kubernetes.io/projected/ed0984e8-4daf-49c8-b7d9-aff614349488-kube-api-access-7cltk\") pod \"odh-model-controller-7998c67dc4-w2cqz\" (UID: \"ed0984e8-4daf-49c8-b7d9-aff614349488\") " pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:07.899223 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.899190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed0984e8-4daf-49c8-b7d9-aff614349488-cert\") pod \"odh-model-controller-7998c67dc4-w2cqz\" (UID: \"ed0984e8-4daf-49c8-b7d9-aff614349488\") " pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:07.899384 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.899252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cltk\" (UniqueName: \"kubernetes.io/projected/ed0984e8-4daf-49c8-b7d9-aff614349488-kube-api-access-7cltk\") pod \"odh-model-controller-7998c67dc4-w2cqz\" (UID: \"ed0984e8-4daf-49c8-b7d9-aff614349488\") " pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:07.901559 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.901530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed0984e8-4daf-49c8-b7d9-aff614349488-cert\") pod \"odh-model-controller-7998c67dc4-w2cqz\" (UID: \"ed0984e8-4daf-49c8-b7d9-aff614349488\") " pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:07.911073 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:07.911051 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cltk\" (UniqueName: \"kubernetes.io/projected/ed0984e8-4daf-49c8-b7d9-aff614349488-kube-api-access-7cltk\") pod \"odh-model-controller-7998c67dc4-w2cqz\" (UID: \"ed0984e8-4daf-49c8-b7d9-aff614349488\") " pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:08.039528 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:08.039446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:08.190554 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:08.190518 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-w2cqz"] Jul 07 19:26:08.193458 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:26:08.193428 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0984e8_4daf_49c8_b7d9_aff614349488.slice/crio-a4415c1358e526ef4c05e5166f153cbcb505d943a019a8b4bdb2f48fdc949a2a WatchSource:0}: Error finding container a4415c1358e526ef4c05e5166f153cbcb505d943a019a8b4bdb2f48fdc949a2a: Status 404 returned error can't find the container with id a4415c1358e526ef4c05e5166f153cbcb505d943a019a8b4bdb2f48fdc949a2a Jul 07 19:26:08.214247 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:08.214218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" event={"ID":"ed0984e8-4daf-49c8-b7d9-aff614349488","Type":"ContainerStarted","Data":"a4415c1358e526ef4c05e5166f153cbcb505d943a019a8b4bdb2f48fdc949a2a"} Jul 07 19:26:12.241691 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:12.241650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" event={"ID":"ed0984e8-4daf-49c8-b7d9-aff614349488","Type":"ContainerStarted","Data":"a51abdb744fc42f0aca25cf621d8a10ca5a25a439b2a5b127796cc08e648038d"} Jul 07 19:26:12.242115 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:12.241752 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:12.265690 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:12.265640 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" podStartSLOduration=2.259814232 podStartE2EDuration="5.265624719s" totalImagesPullingTime="3.005810487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:12.263916719 +0000 UTC m=+288.048901733" watchObservedRunningTime="2026-07-07 19:26:12.265624719 +0000 UTC m=+288.050609718" Jul 07 19:26:14.902806 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.902772 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb"] Jul 07 19:26:14.905600 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.905584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:14.917597 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.917576 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 19:26:14.917740 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.917572 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 19:26:14.917740 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.917583 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b4pmc\"" Jul 07 19:26:14.917740 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.917642 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 19:26:14.939880 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.939846 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb"] Jul 07 19:26:14.960672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.960642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/890176dc-a40d-4d9d-9353-5e28aa1fc083-manager-config\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:14.960672 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.960677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8852n\" (UniqueName: \"kubernetes.io/projected/890176dc-a40d-4d9d-9353-5e28aa1fc083-kube-api-access-8852n\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:14.960862 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.960706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/890176dc-a40d-4d9d-9353-5e28aa1fc083-metrics-cert\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:14.960862 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:14.960758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/890176dc-a40d-4d9d-9353-5e28aa1fc083-cert\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.061762 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.061728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/890176dc-a40d-4d9d-9353-5e28aa1fc083-cert\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.061923 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.061792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/890176dc-a40d-4d9d-9353-5e28aa1fc083-manager-config\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.062005 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.061962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8852n\" (UniqueName: \"kubernetes.io/projected/890176dc-a40d-4d9d-9353-5e28aa1fc083-kube-api-access-8852n\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.062066 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.062040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/890176dc-a40d-4d9d-9353-5e28aa1fc083-metrics-cert\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.062401 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.062381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/890176dc-a40d-4d9d-9353-5e28aa1fc083-manager-config\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.064232 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.064210 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/890176dc-a40d-4d9d-9353-5e28aa1fc083-cert\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.064315 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.064246 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/890176dc-a40d-4d9d-9353-5e28aa1fc083-metrics-cert\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.082699 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.082669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8852n\" (UniqueName: \"kubernetes.io/projected/890176dc-a40d-4d9d-9353-5e28aa1fc083-kube-api-access-8852n\") pod \"lws-controller-manager-57f454c8fb-hm2kb\" (UID: \"890176dc-a40d-4d9d-9353-5e28aa1fc083\") " pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.214433 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.214349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:15.353468 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:15.353439 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb"] Jul 07 19:26:15.356373 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:26:15.356350 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890176dc_a40d_4d9d_9353_5e28aa1fc083.slice/crio-2d2be977b5a45325919f6f2cee9ce152ee8d03caeb0850453d7aae3666ae5a8f WatchSource:0}: Error finding container 2d2be977b5a45325919f6f2cee9ce152ee8d03caeb0850453d7aae3666ae5a8f: Status 404 returned error can't find the container with id 2d2be977b5a45325919f6f2cee9ce152ee8d03caeb0850453d7aae3666ae5a8f Jul 07 19:26:16.263073 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:16.263034 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" event={"ID":"890176dc-a40d-4d9d-9353-5e28aa1fc083","Type":"ContainerStarted","Data":"2d2be977b5a45325919f6f2cee9ce152ee8d03caeb0850453d7aae3666ae5a8f"} Jul 07 19:26:17.269405 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:17.269372 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" event={"ID":"890176dc-a40d-4d9d-9353-5e28aa1fc083","Type":"ContainerStarted","Data":"3195dc6b24508d42af7d6f9788651d486f10e953e093f3a6a6ef80b176be6680"} Jul 07 19:26:17.269797 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:17.269505 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:20.762949 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.762895 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" podStartSLOduration=5.672219503 podStartE2EDuration="6.762878676s" totalImagesPullingTime="1.090659173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:17.396611108 +0000 UTC m=+293.181596108" watchObservedRunningTime="2026-07-07 19:26:20.762878676 +0000 UTC m=+296.547863704" Jul 07 19:26:20.763900 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.763881 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm"] Jul 07 19:26:20.766588 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.766573 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.775491 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.775472 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 19:26:20.778087 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.778070 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 19:26:20.778186 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.778069 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 19:26:20.778186 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.778070 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 19:26:20.779484 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.779468 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 19:26:20.783314 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.783301 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 19:26:20.799216 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.799192 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gg84n\"" Jul 07 19:26:20.802689 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.802664 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm"] Jul 07 19:26:20.912020 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.911989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/992194d3-c7d6-4a92-8643-5a84c4467664-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.912214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.912037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.912214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.912081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/992194d3-c7d6-4a92-8643-5a84c4467664-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.912214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.912107 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.912214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.912124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjpbj\" (UniqueName: \"kubernetes.io/projected/992194d3-c7d6-4a92-8643-5a84c4467664-kube-api-access-jjpbj\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.912214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.912141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/992194d3-c7d6-4a92-8643-5a84c4467664-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:20.912462 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:20.912219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/992194d3-c7d6-4a92-8643-5a84c4467664-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013529 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjpbj\" (UniqueName: \"kubernetes.io/projected/992194d3-c7d6-4a92-8643-5a84c4467664-kube-api-access-jjpbj\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013529 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/992194d3-c7d6-4a92-8643-5a84c4467664-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013529 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013529 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/992194d3-c7d6-4a92-8643-5a84c4467664-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.013964 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.013939 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/992194d3-c7d6-4a92-8643-5a84c4467664-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.015592 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.015564 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.015704 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.015643 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/992194d3-c7d6-4a92-8643-5a84c4467664-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.015763 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.015717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.015763 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.015757 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/992194d3-c7d6-4a92-8643-5a84c4467664-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.048586 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.048551 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjpbj\" (UniqueName: \"kubernetes.io/projected/992194d3-c7d6-4a92-8643-5a84c4467664-kube-api-access-jjpbj\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.056212 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.056187 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/992194d3-c7d6-4a92-8643-5a84c4467664-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7bbzm\" (UID: \"992194d3-c7d6-4a92-8643-5a84c4467664\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.075170 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.075140 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:21.222112 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.221772 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm"] Jul 07 19:26:21.223709 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:26:21.223681 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992194d3_c7d6_4a92_8643_5a84c4467664.slice/crio-34c16ff7a9ae42d0474f88dc85219a939a6b68a82d66ab3a6e09d4d92447b42f WatchSource:0}: Error finding container 34c16ff7a9ae42d0474f88dc85219a939a6b68a82d66ab3a6e09d4d92447b42f: Status 404 returned error can't find the container with id 34c16ff7a9ae42d0474f88dc85219a939a6b68a82d66ab3a6e09d4d92447b42f Jul 07 19:26:21.288634 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:21.288541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" event={"ID":"992194d3-c7d6-4a92-8643-5a84c4467664","Type":"ContainerStarted","Data":"34c16ff7a9ae42d0474f88dc85219a939a6b68a82d66ab3a6e09d4d92447b42f"} Jul 07 19:26:23.248339 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:23.248309 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-w2cqz" Jul 07 19:26:23.748932 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:23.748893 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 19:26:23.749038 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:23.749004 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 19:26:24.306854 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:24.306825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" event={"ID":"992194d3-c7d6-4a92-8643-5a84c4467664","Type":"ContainerStarted","Data":"9a0d84a88f71902759cc4e89acac1b8dfd464af4758901702c58dd201861c843"} Jul 07 19:26:24.307298 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:24.306967 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:26:24.308612 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:24.308587 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-7bbzm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 19:26:24.308736 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:24.308646 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" podUID="992194d3-c7d6-4a92-8643-5a84c4467664" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 19:26:24.343410 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:24.343349 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" podStartSLOduration=1.820542084 podStartE2EDuration="4.343325101s" totalImagesPullingTime="2.522783017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:24.341834476 +0000 UTC m=+300.126819513" watchObservedRunningTime="2026-07-07 19:26:24.343325101 +0000 UTC m=+300.128310107" Jul 07 19:26:24.698077 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:24.697940 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 19:26:25.312113 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:25.312079 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-7bbzm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 19:26:25.313585 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:25.312132 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" podUID="992194d3-c7d6-4a92-8643-5a84c4467664" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 19:26:28.276223 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:28.276193 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57f454c8fb-hm2kb" Jul 07 19:26:28.312741 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:26:28.312714 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bbzm" Jul 07 19:27:24.318280 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.318249 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8"] Jul 07 19:27:24.321142 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.321120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.323498 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.323460 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 19:27:24.324271 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.324250 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 19:27:24.324388 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.324279 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f5gcm\"" Jul 07 19:27:24.333933 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.333905 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8"] Jul 07 19:27:24.434528 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.434496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.434528 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.434539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtx2z\" (UniqueName: \"kubernetes.io/projected/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-kube-api-access-dtx2z\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.434743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.434668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.535348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.535314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtx2z\" (UniqueName: \"kubernetes.io/projected/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-kube-api-access-dtx2z\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.535539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.535378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.535539 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.535408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.535829 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.535807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.535917 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.535839 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.544665 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.544634 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtx2z\" (UniqueName: \"kubernetes.io/projected/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-kube-api-access-dtx2z\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.631734 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.631646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:24.764754 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:24.764733 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8"] Jul 07 19:27:24.766950 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:27:24.766921 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0d2248_2220_4a1d_8166_ac5dcc5e45a6.slice/crio-3703975a30cf0c13288f115e3388ec55b0d2361e147ea0fc26b132459de732a9 WatchSource:0}: Error finding container 3703975a30cf0c13288f115e3388ec55b0d2361e147ea0fc26b132459de732a9: Status 404 returned error can't find the container with id 3703975a30cf0c13288f115e3388ec55b0d2361e147ea0fc26b132459de732a9 Jul 07 19:27:25.137924 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.137890 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59"] Jul 07 19:27:25.140901 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.140885 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.164354 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.164323 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59"] Jul 07 19:27:25.240988 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.240937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.241163 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.241010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjrfl\" (UniqueName: \"kubernetes.io/projected/c0f2e46a-3445-434a-8fc4-15c4e731a881-kube-api-access-bjrfl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.241163 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.241056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.342297 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.342265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.342297 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.342309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjrfl\" (UniqueName: \"kubernetes.io/projected/c0f2e46a-3445-434a-8fc4-15c4e731a881-kube-api-access-bjrfl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.342746 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.342351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.342746 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.342663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.342746 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.342689 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.351695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.351672 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjrfl\" (UniqueName: \"kubernetes.io/projected/c0f2e46a-3445-434a-8fc4-15c4e731a881-kube-api-access-bjrfl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.449913 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.449830 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:25.591841 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.591816 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59"] Jul 07 19:27:25.591917 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.591859 2580 generic.go:356] "Generic (PLEG): container finished" podID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerID="eea401a864d014ca7c23e59319a477d15faac1c8a1f8126532d395a44bdaf466" exitCode=0 Jul 07 19:27:25.591917 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.591880 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" event={"ID":"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6","Type":"ContainerDied","Data":"eea401a864d014ca7c23e59319a477d15faac1c8a1f8126532d395a44bdaf466"} Jul 07 19:27:25.592019 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.591915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" event={"ID":"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6","Type":"ContainerStarted","Data":"3703975a30cf0c13288f115e3388ec55b0d2361e147ea0fc26b132459de732a9"} Jul 07 19:27:25.592477 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.592462 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:27:25.594260 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:27:25.594237 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f2e46a_3445_434a_8fc4_15c4e731a881.slice/crio-4ad9bae573f6a9dc655d64fb27524768045075abdcd241bfeaa7fe4f3adba96e WatchSource:0}: Error finding container 4ad9bae573f6a9dc655d64fb27524768045075abdcd241bfeaa7fe4f3adba96e: Status 404 returned error can't find the container with id 4ad9bae573f6a9dc655d64fb27524768045075abdcd241bfeaa7fe4f3adba96e Jul 07 19:27:25.922121 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.922084 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj"] Jul 07 19:27:25.924855 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.924838 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:25.936124 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:25.936100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj"] Jul 07 19:27:26.048338 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.048304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9b4\" (UniqueName: \"kubernetes.io/projected/bc7699ac-a26e-4a0a-8511-136482e94baf-kube-api-access-rc9b4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.048526 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.048364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.048526 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.048389 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.149367 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.149328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.149367 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.149363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.149581 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.149422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9b4\" (UniqueName: \"kubernetes.io/projected/bc7699ac-a26e-4a0a-8511-136482e94baf-kube-api-access-rc9b4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.149747 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.149719 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.149819 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.149746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.160113 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.160082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9b4\" (UniqueName: \"kubernetes.io/projected/bc7699ac-a26e-4a0a-8511-136482e94baf-kube-api-access-rc9b4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.233500 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.233477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:26.376072 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.376042 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj"] Jul 07 19:27:26.386954 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:27:26.386929 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7699ac_a26e_4a0a_8511_136482e94baf.slice/crio-7dc0c47e0a5b910659c8505e50e3f6c10b3b46032ee4f50d8507dcdd51fcb8d0 WatchSource:0}: Error finding container 7dc0c47e0a5b910659c8505e50e3f6c10b3b46032ee4f50d8507dcdd51fcb8d0: Status 404 returned error can't find the container with id 7dc0c47e0a5b910659c8505e50e3f6c10b3b46032ee4f50d8507dcdd51fcb8d0 Jul 07 19:27:26.512682 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.512651 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl"] Jul 07 19:27:26.516534 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.516517 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.524687 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.524663 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl"] Jul 07 19:27:26.603787 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.603751 2580 generic.go:356] "Generic (PLEG): container finished" podID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerID="f4eaa1d91389245b30aea51f391f26f791d5d7b2c7697d6d7955647787a51b00" exitCode=0 Jul 07 19:27:26.604003 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.603838 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" event={"ID":"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6","Type":"ContainerDied","Data":"f4eaa1d91389245b30aea51f391f26f791d5d7b2c7697d6d7955647787a51b00"} Jul 07 19:27:26.605425 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.605400 2580 generic.go:356] "Generic (PLEG): container finished" podID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerID="5e6f47d6e8ded20d412bbc96a1d65839eaa19d5e2f32d7e08ed012e86c68eed7" exitCode=0 Jul 07 19:27:26.605517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.605493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" event={"ID":"bc7699ac-a26e-4a0a-8511-136482e94baf","Type":"ContainerDied","Data":"5e6f47d6e8ded20d412bbc96a1d65839eaa19d5e2f32d7e08ed012e86c68eed7"} Jul 07 19:27:26.605582 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.605555 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" event={"ID":"bc7699ac-a26e-4a0a-8511-136482e94baf","Type":"ContainerStarted","Data":"7dc0c47e0a5b910659c8505e50e3f6c10b3b46032ee4f50d8507dcdd51fcb8d0"} Jul 07 19:27:26.607081 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.607062 2580 generic.go:356] "Generic (PLEG): container finished" podID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerID="1bcea2a202cbac730413d9ade97300dbaa9db8064dac629e9dbfc2629bb1e45f" exitCode=0 Jul 07 19:27:26.607162 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.607088 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" event={"ID":"c0f2e46a-3445-434a-8fc4-15c4e731a881","Type":"ContainerDied","Data":"1bcea2a202cbac730413d9ade97300dbaa9db8064dac629e9dbfc2629bb1e45f"} Jul 07 19:27:26.607162 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.607130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" event={"ID":"c0f2e46a-3445-434a-8fc4-15c4e731a881","Type":"ContainerStarted","Data":"4ad9bae573f6a9dc655d64fb27524768045075abdcd241bfeaa7fe4f3adba96e"} Jul 07 19:27:26.654392 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.654368 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.654472 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.654417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvh6\" (UniqueName: \"kubernetes.io/projected/c20f4295-f762-4eb4-a793-bcff05004ede-kube-api-access-zsvh6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.654520 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.654489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.755109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.755009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvh6\" (UniqueName: \"kubernetes.io/projected/c20f4295-f762-4eb4-a793-bcff05004ede-kube-api-access-zsvh6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.755109 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.755069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.755331 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.755211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.755468 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.755443 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.755537 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.755498 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.765138 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.765118 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvh6\" (UniqueName: \"kubernetes.io/projected/c20f4295-f762-4eb4-a793-bcff05004ede-kube-api-access-zsvh6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.859827 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.859791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:26.989706 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:26.989676 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl"] Jul 07 19:27:26.994576 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:27:26.994547 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20f4295_f762_4eb4_a793_bcff05004ede.slice/crio-33f0b4b122fefd4df0c655cbff88ba08f9f7fd1e959dfa7b50a054c412aefb89 WatchSource:0}: Error finding container 33f0b4b122fefd4df0c655cbff88ba08f9f7fd1e959dfa7b50a054c412aefb89: Status 404 returned error can't find the container with id 33f0b4b122fefd4df0c655cbff88ba08f9f7fd1e959dfa7b50a054c412aefb89 Jul 07 19:27:27.613375 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.613346 2580 generic.go:356] "Generic (PLEG): container finished" podID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerID="ed606056f904112a64d6691c48ec8741ff0c6f0e30adb8d4b6a03b506a97f9ca" exitCode=0 Jul 07 19:27:27.613806 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.613419 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" event={"ID":"bc7699ac-a26e-4a0a-8511-136482e94baf","Type":"ContainerDied","Data":"ed606056f904112a64d6691c48ec8741ff0c6f0e30adb8d4b6a03b506a97f9ca"} Jul 07 19:27:27.615110 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.615006 2580 generic.go:356] "Generic (PLEG): container finished" podID="c20f4295-f762-4eb4-a793-bcff05004ede" containerID="d4b99c1ec60322293ea0e3f6f65b853b7a972f80624d7425bba4950ea0e45e67" exitCode=0 Jul 07 19:27:27.615110 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.615101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" event={"ID":"c20f4295-f762-4eb4-a793-bcff05004ede","Type":"ContainerDied","Data":"d4b99c1ec60322293ea0e3f6f65b853b7a972f80624d7425bba4950ea0e45e67"} Jul 07 19:27:27.615230 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.615135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" event={"ID":"c20f4295-f762-4eb4-a793-bcff05004ede","Type":"ContainerStarted","Data":"33f0b4b122fefd4df0c655cbff88ba08f9f7fd1e959dfa7b50a054c412aefb89"} Jul 07 19:27:27.616944 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.616922 2580 generic.go:356] "Generic (PLEG): container finished" podID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerID="27e6ea17ae1a0cd42620c8293a3b63686f926e2d9fe358b160c26477823ac371" exitCode=0 Jul 07 19:27:27.617084 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.617063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" event={"ID":"c0f2e46a-3445-434a-8fc4-15c4e731a881","Type":"ContainerDied","Data":"27e6ea17ae1a0cd42620c8293a3b63686f926e2d9fe358b160c26477823ac371"} Jul 07 19:27:27.619444 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.619425 2580 generic.go:356] "Generic (PLEG): container finished" podID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerID="cf45f007b1e30dc2664ec0e4d3bb041ee7329072e6dd3023007642bcf7929eaf" exitCode=0 Jul 07 19:27:27.619537 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:27.619474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" event={"ID":"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6","Type":"ContainerDied","Data":"cf45f007b1e30dc2664ec0e4d3bb041ee7329072e6dd3023007642bcf7929eaf"} Jul 07 19:27:28.626029 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.625990 2580 generic.go:356] "Generic (PLEG): container finished" podID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerID="45efcd4f25485293f986b5e5d13e5bf696624a8f850381ec4be5d1d5ac25ee4f" exitCode=0 Jul 07 19:27:28.626435 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.626080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" event={"ID":"c0f2e46a-3445-434a-8fc4-15c4e731a881","Type":"ContainerDied","Data":"45efcd4f25485293f986b5e5d13e5bf696624a8f850381ec4be5d1d5ac25ee4f"} Jul 07 19:27:28.628115 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.628092 2580 generic.go:356] "Generic (PLEG): container finished" podID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerID="f3d61bbd8389052c20eaa743be84b3fa06ef80099e5ebf88f47197eaefc070d5" exitCode=0 Jul 07 19:27:28.628227 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.628163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" event={"ID":"bc7699ac-a26e-4a0a-8511-136482e94baf","Type":"ContainerDied","Data":"f3d61bbd8389052c20eaa743be84b3fa06ef80099e5ebf88f47197eaefc070d5"} Jul 07 19:27:28.629850 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.629831 2580 generic.go:356] "Generic (PLEG): container finished" podID="c20f4295-f762-4eb4-a793-bcff05004ede" containerID="5664f103b9b8b2f9c168bc0933f4299680a6cca60db3f49572ae4aa11c945113" exitCode=0 Jul 07 19:27:28.629940 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.629898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" event={"ID":"c20f4295-f762-4eb4-a793-bcff05004ede","Type":"ContainerDied","Data":"5664f103b9b8b2f9c168bc0933f4299680a6cca60db3f49572ae4aa11c945113"} Jul 07 19:27:28.771457 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.771431 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:28.873121 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.873089 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-util\") pod \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " Jul 07 19:27:28.873294 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.873149 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-bundle\") pod \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " Jul 07 19:27:28.873294 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.873180 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtx2z\" (UniqueName: \"kubernetes.io/projected/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-kube-api-access-dtx2z\") pod \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\" (UID: \"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6\") " Jul 07 19:27:28.873668 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.873643 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-bundle" (OuterVolumeSpecName: "bundle") pod "8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" (UID: "8d0d2248-2220-4a1d-8166-ac5dcc5e45a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:28.875426 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.875406 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-kube-api-access-dtx2z" (OuterVolumeSpecName: "kube-api-access-dtx2z") pod "8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" (UID: "8d0d2248-2220-4a1d-8166-ac5dcc5e45a6"). InnerVolumeSpecName "kube-api-access-dtx2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:27:28.878455 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.878404 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-util" (OuterVolumeSpecName: "util") pod "8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" (UID: "8d0d2248-2220-4a1d-8166-ac5dcc5e45a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:28.974076 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.974041 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:28.974076 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.974071 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtx2z\" (UniqueName: \"kubernetes.io/projected/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-kube-api-access-dtx2z\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:28.974265 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:28.974087 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d0d2248-2220-4a1d-8166-ac5dcc5e45a6-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:29.637421 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.637386 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" Jul 07 19:27:29.637851 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.637388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8" event={"ID":"8d0d2248-2220-4a1d-8166-ac5dcc5e45a6","Type":"ContainerDied","Data":"3703975a30cf0c13288f115e3388ec55b0d2361e147ea0fc26b132459de732a9"} Jul 07 19:27:29.637851 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.637500 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3703975a30cf0c13288f115e3388ec55b0d2361e147ea0fc26b132459de732a9" Jul 07 19:27:29.639492 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.639464 2580 generic.go:356] "Generic (PLEG): container finished" podID="c20f4295-f762-4eb4-a793-bcff05004ede" containerID="86101cbea2bfef55bcb9d0fb78f18867c7fbfefe7b57d45e25300a7557de1403" exitCode=0 Jul 07 19:27:29.639633 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.639528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" event={"ID":"c20f4295-f762-4eb4-a793-bcff05004ede","Type":"ContainerDied","Data":"86101cbea2bfef55bcb9d0fb78f18867c7fbfefe7b57d45e25300a7557de1403"} Jul 07 19:27:29.794284 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.794260 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:29.796957 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.796937 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:29.882222 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882128 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc9b4\" (UniqueName: \"kubernetes.io/projected/bc7699ac-a26e-4a0a-8511-136482e94baf-kube-api-access-rc9b4\") pod \"bc7699ac-a26e-4a0a-8511-136482e94baf\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " Jul 07 19:27:29.882222 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882190 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-util\") pod \"bc7699ac-a26e-4a0a-8511-136482e94baf\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " Jul 07 19:27:29.882222 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882207 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjrfl\" (UniqueName: \"kubernetes.io/projected/c0f2e46a-3445-434a-8fc4-15c4e731a881-kube-api-access-bjrfl\") pod \"c0f2e46a-3445-434a-8fc4-15c4e731a881\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " Jul 07 19:27:29.882507 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882322 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-bundle\") pod \"c0f2e46a-3445-434a-8fc4-15c4e731a881\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " Jul 07 19:27:29.882507 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882351 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-util\") pod \"c0f2e46a-3445-434a-8fc4-15c4e731a881\" (UID: \"c0f2e46a-3445-434a-8fc4-15c4e731a881\") " Jul 07 19:27:29.882507 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882368 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-bundle\") pod \"bc7699ac-a26e-4a0a-8511-136482e94baf\" (UID: \"bc7699ac-a26e-4a0a-8511-136482e94baf\") " Jul 07 19:27:29.882890 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.882861 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-bundle" (OuterVolumeSpecName: "bundle") pod "c0f2e46a-3445-434a-8fc4-15c4e731a881" (UID: "c0f2e46a-3445-434a-8fc4-15c4e731a881"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:29.883200 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.883163 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-bundle" (OuterVolumeSpecName: "bundle") pod "bc7699ac-a26e-4a0a-8511-136482e94baf" (UID: "bc7699ac-a26e-4a0a-8511-136482e94baf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:29.884457 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.884431 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f2e46a-3445-434a-8fc4-15c4e731a881-kube-api-access-bjrfl" (OuterVolumeSpecName: "kube-api-access-bjrfl") pod "c0f2e46a-3445-434a-8fc4-15c4e731a881" (UID: "c0f2e46a-3445-434a-8fc4-15c4e731a881"). InnerVolumeSpecName "kube-api-access-bjrfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:27:29.884554 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.884489 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc7699ac-a26e-4a0a-8511-136482e94baf-kube-api-access-rc9b4" (OuterVolumeSpecName: "kube-api-access-rc9b4") pod "bc7699ac-a26e-4a0a-8511-136482e94baf" (UID: "bc7699ac-a26e-4a0a-8511-136482e94baf"). InnerVolumeSpecName "kube-api-access-rc9b4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:27:29.887920 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.887896 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-util" (OuterVolumeSpecName: "util") pod "c0f2e46a-3445-434a-8fc4-15c4e731a881" (UID: "c0f2e46a-3445-434a-8fc4-15c4e731a881"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:29.888185 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.888168 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-util" (OuterVolumeSpecName: "util") pod "bc7699ac-a26e-4a0a-8511-136482e94baf" (UID: "bc7699ac-a26e-4a0a-8511-136482e94baf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:29.983370 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.983321 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:29.983370 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.983361 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f2e46a-3445-434a-8fc4-15c4e731a881-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:29.983370 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.983372 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:29.983370 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.983380 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc9b4\" (UniqueName: \"kubernetes.io/projected/bc7699ac-a26e-4a0a-8511-136482e94baf-kube-api-access-rc9b4\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:29.983370 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.983390 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc7699ac-a26e-4a0a-8511-136482e94baf-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:29.983659 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:29.983399 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjrfl\" (UniqueName: \"kubernetes.io/projected/c0f2e46a-3445-434a-8fc4-15c4e731a881-kube-api-access-bjrfl\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:30.647022 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.646994 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" Jul 07 19:27:30.647022 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.647007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj" event={"ID":"bc7699ac-a26e-4a0a-8511-136482e94baf","Type":"ContainerDied","Data":"7dc0c47e0a5b910659c8505e50e3f6c10b3b46032ee4f50d8507dcdd51fcb8d0"} Jul 07 19:27:30.647544 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.647054 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dc0c47e0a5b910659c8505e50e3f6c10b3b46032ee4f50d8507dcdd51fcb8d0" Jul 07 19:27:30.648784 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.648767 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" Jul 07 19:27:30.648916 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.648795 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59" event={"ID":"c0f2e46a-3445-434a-8fc4-15c4e731a881","Type":"ContainerDied","Data":"4ad9bae573f6a9dc655d64fb27524768045075abdcd241bfeaa7fe4f3adba96e"} Jul 07 19:27:30.648916 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.648830 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ad9bae573f6a9dc655d64fb27524768045075abdcd241bfeaa7fe4f3adba96e" Jul 07 19:27:30.784912 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.784890 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:30.891354 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.891316 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-util\") pod \"c20f4295-f762-4eb4-a793-bcff05004ede\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " Jul 07 19:27:30.891534 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.891363 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-bundle\") pod \"c20f4295-f762-4eb4-a793-bcff05004ede\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " Jul 07 19:27:30.891534 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.891396 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsvh6\" (UniqueName: \"kubernetes.io/projected/c20f4295-f762-4eb4-a793-bcff05004ede-kube-api-access-zsvh6\") pod \"c20f4295-f762-4eb4-a793-bcff05004ede\" (UID: \"c20f4295-f762-4eb4-a793-bcff05004ede\") " Jul 07 19:27:30.891989 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.891947 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-bundle" (OuterVolumeSpecName: "bundle") pod "c20f4295-f762-4eb4-a793-bcff05004ede" (UID: "c20f4295-f762-4eb4-a793-bcff05004ede"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:30.893527 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.893495 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c20f4295-f762-4eb4-a793-bcff05004ede-kube-api-access-zsvh6" (OuterVolumeSpecName: "kube-api-access-zsvh6") pod "c20f4295-f762-4eb4-a793-bcff05004ede" (UID: "c20f4295-f762-4eb4-a793-bcff05004ede"). InnerVolumeSpecName "kube-api-access-zsvh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:27:30.896910 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.896874 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-util" (OuterVolumeSpecName: "util") pod "c20f4295-f762-4eb4-a793-bcff05004ede" (UID: "c20f4295-f762-4eb4-a793-bcff05004ede"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:30.992337 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.992305 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:30.992337 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.992333 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsvh6\" (UniqueName: \"kubernetes.io/projected/c20f4295-f762-4eb4-a793-bcff05004ede-kube-api-access-zsvh6\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:30.992536 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:30.992348 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c20f4295-f762-4eb4-a793-bcff05004ede-util\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:31.655094 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:31.655062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" event={"ID":"c20f4295-f762-4eb4-a793-bcff05004ede","Type":"ContainerDied","Data":"33f0b4b122fefd4df0c655cbff88ba08f9f7fd1e959dfa7b50a054c412aefb89"} Jul 07 19:27:31.655577 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:31.655118 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl" Jul 07 19:27:31.655577 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:27:31.655141 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33f0b4b122fefd4df0c655cbff88ba08f9f7fd1e959dfa7b50a054c412aefb89" Jul 07 19:28:38.044397 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044360 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044657 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044667 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044675 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044680 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044690 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044695 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044707 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044713 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044718 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044723 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044729 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044734 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044740 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044745 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044750 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044754 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="pull" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044764 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044769 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="util" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044775 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044780 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044785 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044790 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044795 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="extract" Jul 07 19:28:38.044837 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044800 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="extract" Jul 07 19:28:38.045581 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044855 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d0d2248-2220-4a1d-8166-ac5dcc5e45a6" containerName="extract" Jul 07 19:28:38.045581 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044862 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc7699ac-a26e-4a0a-8511-136482e94baf" containerName="extract" Jul 07 19:28:38.045581 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044870 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c20f4295-f762-4eb4-a793-bcff05004ede" containerName="extract" Jul 07 19:28:38.045581 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.044876 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f2e46a-3445-434a-8fc4-15c4e731a881" containerName="extract" Jul 07 19:28:38.047257 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.047242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.050127 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.050108 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f5gcm\"" Jul 07 19:28:38.050683 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.050663 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 19:28:38.050927 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.050909 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 19:28:38.051043 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.050909 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 19:28:38.060318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.060297 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:38.112623 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.112588 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g54h\" (UniqueName: \"kubernetes.io/projected/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-kube-api-access-8g54h\") pod \"limitador-limitador-64c94fd64f-fqtv7\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.112786 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.112637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-config-file\") pod \"limitador-limitador-64c94fd64f-fqtv7\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.161474 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.161444 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:38.213863 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.213837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-config-file\") pod \"limitador-limitador-64c94fd64f-fqtv7\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.214088 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.213894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g54h\" (UniqueName: \"kubernetes.io/projected/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-kube-api-access-8g54h\") pod \"limitador-limitador-64c94fd64f-fqtv7\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.214532 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.214508 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-config-file\") pod \"limitador-limitador-64c94fd64f-fqtv7\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.228806 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.228784 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g54h\" (UniqueName: \"kubernetes.io/projected/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-kube-api-access-8g54h\") pod \"limitador-limitador-64c94fd64f-fqtv7\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.357719 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.357686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:38.494037 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.494010 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:38.495405 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:28:38.495377 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5d48f6_a2d1_43cf_a935_e61d7ce9dd5b.slice/crio-597ac85b3e2a0bf19f1ac8d24631f18313f31f01be707a21540e79768d2a3758 WatchSource:0}: Error finding container 597ac85b3e2a0bf19f1ac8d24631f18313f31f01be707a21540e79768d2a3758: Status 404 returned error can't find the container with id 597ac85b3e2a0bf19f1ac8d24631f18313f31f01be707a21540e79768d2a3758 Jul 07 19:28:38.979257 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:38.979227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" event={"ID":"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b","Type":"ContainerStarted","Data":"597ac85b3e2a0bf19f1ac8d24631f18313f31f01be707a21540e79768d2a3758"} Jul 07 19:28:41.998922 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:41.998882 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" event={"ID":"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b","Type":"ContainerStarted","Data":"a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d"} Jul 07 19:28:41.999328 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:41.999028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:42.022355 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:42.022310 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" podStartSLOduration=1.4288440310000001 podStartE2EDuration="4.02229404s" totalImagesPullingTime="2.593450009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:28:42.022281343 +0000 UTC m=+437.807266343" watchObservedRunningTime="2026-07-07 19:28:42.02229404 +0000 UTC m=+437.807279043" Jul 07 19:28:52.945196 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:52.945115 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:52.945671 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:52.945292 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" podUID="bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" containerName="limitador" containerID="cri-o://a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d" gracePeriod=30 Jul 07 19:28:52.946036 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:52.945943 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:53.483890 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.483867 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:53.660250 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.660211 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g54h\" (UniqueName: \"kubernetes.io/projected/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-kube-api-access-8g54h\") pod \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " Jul 07 19:28:53.660250 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.660249 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-config-file\") pod \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\" (UID: \"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b\") " Jul 07 19:28:53.660644 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.660614 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-config-file" (OuterVolumeSpecName: "config-file") pod "bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" (UID: "bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:28:53.662264 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.662236 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-kube-api-access-8g54h" (OuterVolumeSpecName: "kube-api-access-8g54h") pod "bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" (UID: "bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b"). InnerVolumeSpecName "kube-api-access-8g54h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:28:53.761715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.761683 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g54h\" (UniqueName: \"kubernetes.io/projected/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-kube-api-access-8g54h\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:53.761715 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:53.761709 2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b-config-file\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:54.053578 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.053543 2580 generic.go:356] "Generic (PLEG): container finished" podID="bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" containerID="a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d" exitCode=0 Jul 07 19:28:54.054049 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.053585 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" event={"ID":"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b","Type":"ContainerDied","Data":"a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d"} Jul 07 19:28:54.054049 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.053615 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" Jul 07 19:28:54.054049 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.053635 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqtv7" event={"ID":"bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b","Type":"ContainerDied","Data":"597ac85b3e2a0bf19f1ac8d24631f18313f31f01be707a21540e79768d2a3758"} Jul 07 19:28:54.054049 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.053666 2580 scope.go:117] "RemoveContainer" containerID="a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d" Jul 07 19:28:54.062911 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.062893 2580 scope.go:117] "RemoveContainer" containerID="a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d" Jul 07 19:28:54.063228 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:28:54.063210 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d\": container with ID starting with a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d not found: ID does not exist" containerID="a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d" Jul 07 19:28:54.063290 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.063237 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d"} err="failed to get container status \"a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d\": rpc error: code = NotFound desc = could not find container \"a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d\": container with ID starting with a54ccde0ac084ec9af4b0a147908f5dcee8a5354cdd99c3c0f8ae552e307679d not found: ID does not exist" Jul 07 19:28:54.081601 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.081576 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:54.084071 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.084047 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqtv7"] Jul 07 19:28:54.437414 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.437334 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-bcq9d"] Jul 07 19:28:54.437632 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.437620 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" containerName="limitador" Jul 07 19:28:54.437681 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.437633 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" containerName="limitador" Jul 07 19:28:54.437734 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.437696 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" containerName="limitador" Jul 07 19:28:54.440525 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.440507 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.442967 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.442946 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 19:28:54.443106 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.443043 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-942lx\"" Jul 07 19:28:54.450037 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.450015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-bcq9d"] Jul 07 19:28:54.567475 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.567446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfxtw\" (UniqueName: \"kubernetes.io/projected/d5ccdece-a292-4aa6-810d-9364f00ef4d8-kube-api-access-bfxtw\") pod \"postgres-69fb4594cf-bcq9d\" (UID: \"d5ccdece-a292-4aa6-810d-9364f00ef4d8\") " pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.567646 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.567505 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d5ccdece-a292-4aa6-810d-9364f00ef4d8-data\") pod \"postgres-69fb4594cf-bcq9d\" (UID: \"d5ccdece-a292-4aa6-810d-9364f00ef4d8\") " pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.668416 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.668386 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d5ccdece-a292-4aa6-810d-9364f00ef4d8-data\") pod \"postgres-69fb4594cf-bcq9d\" (UID: \"d5ccdece-a292-4aa6-810d-9364f00ef4d8\") " pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.668605 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.668442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfxtw\" (UniqueName: \"kubernetes.io/projected/d5ccdece-a292-4aa6-810d-9364f00ef4d8-kube-api-access-bfxtw\") pod \"postgres-69fb4594cf-bcq9d\" (UID: \"d5ccdece-a292-4aa6-810d-9364f00ef4d8\") " pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.668789 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.668769 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d5ccdece-a292-4aa6-810d-9364f00ef4d8-data\") pod \"postgres-69fb4594cf-bcq9d\" (UID: \"d5ccdece-a292-4aa6-810d-9364f00ef4d8\") " pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.678462 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.678428 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfxtw\" (UniqueName: \"kubernetes.io/projected/d5ccdece-a292-4aa6-810d-9364f00ef4d8-kube-api-access-bfxtw\") pod \"postgres-69fb4594cf-bcq9d\" (UID: \"d5ccdece-a292-4aa6-810d-9364f00ef4d8\") " pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.752263 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.752180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:28:54.884379 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.884345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-bcq9d"] Jul 07 19:28:54.887000 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:28:54.886960 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ccdece_a292_4aa6_810d_9364f00ef4d8.slice/crio-334bbc77d56090e286c20738425b08d67626424e092626b3171938bf4e629867 WatchSource:0}: Error finding container 334bbc77d56090e286c20738425b08d67626424e092626b3171938bf4e629867: Status 404 returned error can't find the container with id 334bbc77d56090e286c20738425b08d67626424e092626b3171938bf4e629867 Jul 07 19:28:54.915417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:54.915383 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b" path="/var/lib/kubelet/pods/bb5d48f6-a2d1-43cf-a935-e61d7ce9dd5b/volumes" Jul 07 19:28:55.062477 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:28:55.062444 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-bcq9d" event={"ID":"d5ccdece-a292-4aa6-810d-9364f00ef4d8","Type":"ContainerStarted","Data":"334bbc77d56090e286c20738425b08d67626424e092626b3171938bf4e629867"} Jul 07 19:29:01.097750 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:01.097708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-bcq9d" event={"ID":"d5ccdece-a292-4aa6-810d-9364f00ef4d8","Type":"ContainerStarted","Data":"4f4090b053927b4b6978740622d6abc723b26830a206f50707d291acb6b0a5c7"} Jul 07 19:29:01.098155 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:01.097776 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:29:01.116490 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:01.116448 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-bcq9d" podStartSLOduration=1.873825018 podStartE2EDuration="7.116432607s" totalImagesPullingTime="5.242607589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:01.115696951 +0000 UTC m=+456.900681964" watchObservedRunningTime="2026-07-07 19:29:01.116432607 +0000 UTC m=+456.901417606" Jul 07 19:29:07.132113 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:07.132084 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-bcq9d" Jul 07 19:29:14.345759 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.345724 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c5f9dc498-mkpdm"] Jul 07 19:29:14.370794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.370766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5f9dc498-mkpdm"] Jul 07 19:29:14.370997 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.370877 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:14.373695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.373673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 19:29:14.373695 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.373687 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 19:29:14.374549 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.374529 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wrh4h\"" Jul 07 19:29:14.418799 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.418769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:14.418953 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.418817 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp94f\" (UniqueName: \"kubernetes.io/projected/ad21028d-d0cb-4c30-9009-1d31ba1341dd-kube-api-access-pp94f\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:14.519873 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.519838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:14.520060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.519890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp94f\" (UniqueName: \"kubernetes.io/projected/ad21028d-d0cb-4c30-9009-1d31ba1341dd-kube-api-access-pp94f\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:14.520060 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:29:14.519999 2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 19:29:14.520060 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:29:14.520059 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert podName:ad21028d-d0cb-4c30-9009-1d31ba1341dd nodeName:}" failed. No retries permitted until 2026-07-07 19:29:15.020041753 +0000 UTC m=+470.805026731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert") pod "maas-controller-5c5f9dc498-mkpdm" (UID: "ad21028d-d0cb-4c30-9009-1d31ba1341dd") : secret "maas-controller-webhook-cert" not found Jul 07 19:29:14.530548 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.530525 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp94f\" (UniqueName: \"kubernetes.io/projected/ad21028d-d0cb-4c30-9009-1d31ba1341dd-kube-api-access-pp94f\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:14.550244 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.550217 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-754f59f895-tj24b"] Jul 07 19:29:14.552812 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.552795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.564156 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.564134 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-754f59f895-tj24b"] Jul 07 19:29:14.621121 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.621040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74z6m\" (UniqueName: \"kubernetes.io/projected/4e2b80a5-3083-431e-9618-f750169424f0-kube-api-access-74z6m\") pod \"maas-controller-754f59f895-tj24b\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.621121 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.621087 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e2b80a5-3083-431e-9618-f750169424f0-webhook-cert\") pod \"maas-controller-754f59f895-tj24b\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.721716 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.721683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e2b80a5-3083-431e-9618-f750169424f0-webhook-cert\") pod \"maas-controller-754f59f895-tj24b\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.721885 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.721766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74z6m\" (UniqueName: \"kubernetes.io/projected/4e2b80a5-3083-431e-9618-f750169424f0-kube-api-access-74z6m\") pod \"maas-controller-754f59f895-tj24b\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.724082 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.724057 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e2b80a5-3083-431e-9618-f750169424f0-webhook-cert\") pod \"maas-controller-754f59f895-tj24b\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.732039 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.732019 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74z6m\" (UniqueName: \"kubernetes.io/projected/4e2b80a5-3083-431e-9618-f750169424f0-kube-api-access-74z6m\") pod \"maas-controller-754f59f895-tj24b\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.862512 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.862481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:14.991469 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:14.991445 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-754f59f895-tj24b"] Jul 07 19:29:14.993372 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:29:14.993346 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2b80a5_3083_431e_9618_f750169424f0.slice/crio-21a5073747ab4d7b2cb576297fd86a7378595a3cee5b49518718f9a154404fd7 WatchSource:0}: Error finding container 21a5073747ab4d7b2cb576297fd86a7378595a3cee5b49518718f9a154404fd7: Status 404 returned error can't find the container with id 21a5073747ab4d7b2cb576297fd86a7378595a3cee5b49518718f9a154404fd7 Jul 07 19:29:15.024844 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:15.024821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:15.027172 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:15.027156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert\") pod \"maas-controller-5c5f9dc498-mkpdm\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:15.168000 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:15.167913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754f59f895-tj24b" event={"ID":"4e2b80a5-3083-431e-9618-f750169424f0","Type":"ContainerStarted","Data":"21a5073747ab4d7b2cb576297fd86a7378595a3cee5b49518718f9a154404fd7"} Jul 07 19:29:15.281353 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:15.281315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:15.412906 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:15.412883 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5f9dc498-mkpdm"] Jul 07 19:29:15.414750 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:29:15.414723 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad21028d_d0cb_4c30_9009_1d31ba1341dd.slice/crio-fb3388133e13c56b8effbf769c90c92031f79479ea7c414fc66ec5e06af07ba1 WatchSource:0}: Error finding container fb3388133e13c56b8effbf769c90c92031f79479ea7c414fc66ec5e06af07ba1: Status 404 returned error can't find the container with id fb3388133e13c56b8effbf769c90c92031f79479ea7c414fc66ec5e06af07ba1 Jul 07 19:29:16.180600 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:16.180560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" event={"ID":"ad21028d-d0cb-4c30-9009-1d31ba1341dd","Type":"ContainerStarted","Data":"fb3388133e13c56b8effbf769c90c92031f79479ea7c414fc66ec5e06af07ba1"} Jul 07 19:29:18.193549 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:18.193511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" event={"ID":"ad21028d-d0cb-4c30-9009-1d31ba1341dd","Type":"ContainerStarted","Data":"13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e"} Jul 07 19:29:18.194019 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:18.193635 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:18.195048 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:18.195027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754f59f895-tj24b" event={"ID":"4e2b80a5-3083-431e-9618-f750169424f0","Type":"ContainerStarted","Data":"96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177"} Jul 07 19:29:18.195171 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:18.195131 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:18.213834 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:18.213785 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" podStartSLOduration=1.9762024459999998 podStartE2EDuration="4.213772457s" totalImagesPullingTime="2.237570011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:18.211567516 +0000 UTC m=+473.996552510" watchObservedRunningTime="2026-07-07 19:29:18.213772457 +0000 UTC m=+473.998757457" Jul 07 19:29:18.231650 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:18.231605 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-754f59f895-tj24b" podStartSLOduration=1.5833799389999998 podStartE2EDuration="4.231591982s" totalImagesPullingTime="2.648212043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:18.230155 +0000 UTC m=+474.015140002" watchObservedRunningTime="2026-07-07 19:29:18.231591982 +0000 UTC m=+474.016576982" Jul 07 19:29:29.206244 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.206215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:29.206643 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.206289 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:29:29.283188 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.283156 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c5f9dc498-mkpdm"] Jul 07 19:29:29.283417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.283391 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" podUID="ad21028d-d0cb-4c30-9009-1d31ba1341dd" containerName="manager" containerID="cri-o://13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e" gracePeriod=30 Jul 07 19:29:29.524524 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.524503 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:29.647417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.647379 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert\") pod \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " Jul 07 19:29:29.647605 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.647444 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp94f\" (UniqueName: \"kubernetes.io/projected/ad21028d-d0cb-4c30-9009-1d31ba1341dd-kube-api-access-pp94f\") pod \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\" (UID: \"ad21028d-d0cb-4c30-9009-1d31ba1341dd\") " Jul 07 19:29:29.649590 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.649561 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ad21028d-d0cb-4c30-9009-1d31ba1341dd" (UID: "ad21028d-d0cb-4c30-9009-1d31ba1341dd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:29:29.649696 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.649598 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad21028d-d0cb-4c30-9009-1d31ba1341dd-kube-api-access-pp94f" (OuterVolumeSpecName: "kube-api-access-pp94f") pod "ad21028d-d0cb-4c30-9009-1d31ba1341dd" (UID: "ad21028d-d0cb-4c30-9009-1d31ba1341dd"). InnerVolumeSpecName "kube-api-access-pp94f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:29:29.748130 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.748095 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp94f\" (UniqueName: \"kubernetes.io/projected/ad21028d-d0cb-4c30-9009-1d31ba1341dd-kube-api-access-pp94f\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:29:29.748130 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:29.748125 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad21028d-d0cb-4c30-9009-1d31ba1341dd-webhook-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:29:30.253634 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.253601 2580 generic.go:356] "Generic (PLEG): container finished" podID="ad21028d-d0cb-4c30-9009-1d31ba1341dd" containerID="13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e" exitCode=0 Jul 07 19:29:30.254147 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.253659 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" Jul 07 19:29:30.254147 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.253698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" event={"ID":"ad21028d-d0cb-4c30-9009-1d31ba1341dd","Type":"ContainerDied","Data":"13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e"} Jul 07 19:29:30.254147 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.253756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5f9dc498-mkpdm" event={"ID":"ad21028d-d0cb-4c30-9009-1d31ba1341dd","Type":"ContainerDied","Data":"fb3388133e13c56b8effbf769c90c92031f79479ea7c414fc66ec5e06af07ba1"} Jul 07 19:29:30.254147 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.253779 2580 scope.go:117] "RemoveContainer" containerID="13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e" Jul 07 19:29:30.262660 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.262642 2580 scope.go:117] "RemoveContainer" containerID="13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e" Jul 07 19:29:30.262931 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:29:30.262909 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e\": container with ID starting with 13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e not found: ID does not exist" containerID="13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e" Jul 07 19:29:30.263070 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.262939 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e"} err="failed to get container status \"13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e\": rpc error: code = NotFound desc = could not find container \"13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e\": container with ID starting with 13cef1a2d13391826cfab4c343e8c7705189706ab3aea6ef4e03922e242f465e not found: ID does not exist" Jul 07 19:29:30.275842 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.275807 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c5f9dc498-mkpdm"] Jul 07 19:29:30.281057 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.281033 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c5f9dc498-mkpdm"] Jul 07 19:29:30.915601 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:30.915566 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad21028d-d0cb-4c30-9009-1d31ba1341dd" path="/var/lib/kubelet/pods/ad21028d-d0cb-4c30-9009-1d31ba1341dd/volumes" Jul 07 19:29:50.933465 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.933428 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6d85ff795b-skvnl"] Jul 07 19:29:50.934055 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.933864 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad21028d-d0cb-4c30-9009-1d31ba1341dd" containerName="manager" Jul 07 19:29:50.934055 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.933882 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad21028d-d0cb-4c30-9009-1d31ba1341dd" containerName="manager" Jul 07 19:29:50.934055 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.934011 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad21028d-d0cb-4c30-9009-1d31ba1341dd" containerName="manager" Jul 07 19:29:50.936789 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.936768 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:50.939867 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.939847 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:29:50.940430 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.940415 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 19:29:50.954342 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:50.954318 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6d85ff795b-skvnl"] Jul 07 19:29:51.029074 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.029037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124bc3ca-de48-4066-9b81-c4c4942e40be-maas-api-tls\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.029235 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.029097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrn9h\" (UniqueName: \"kubernetes.io/projected/124bc3ca-de48-4066-9b81-c4c4942e40be-kube-api-access-mrn9h\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.029235 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.029134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124bc3ca-de48-4066-9b81-c4c4942e40be-openshift-service-ca\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.130578 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.130545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124bc3ca-de48-4066-9b81-c4c4942e40be-maas-api-tls\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.130756 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.130694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrn9h\" (UniqueName: \"kubernetes.io/projected/124bc3ca-de48-4066-9b81-c4c4942e40be-kube-api-access-mrn9h\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.130756 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.130736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124bc3ca-de48-4066-9b81-c4c4942e40be-openshift-service-ca\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.131422 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.131403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124bc3ca-de48-4066-9b81-c4c4942e40be-openshift-service-ca\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.133067 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.133051 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124bc3ca-de48-4066-9b81-c4c4942e40be-maas-api-tls\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.141689 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.141661 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrn9h\" (UniqueName: \"kubernetes.io/projected/124bc3ca-de48-4066-9b81-c4c4942e40be-kube-api-access-mrn9h\") pod \"maas-api-6d85ff795b-skvnl\" (UID: \"124bc3ca-de48-4066-9b81-c4c4942e40be\") " pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.247107 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.246903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:51.382691 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.382665 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6d85ff795b-skvnl"] Jul 07 19:29:51.384838 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:29:51.384815 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124bc3ca_de48_4066_9b81_c4c4942e40be.slice/crio-a3acfbf6a85079a60e3d7def6419c2394bef8e039f428e42d1b2fb05ed068295 WatchSource:0}: Error finding container a3acfbf6a85079a60e3d7def6419c2394bef8e039f428e42d1b2fb05ed068295: Status 404 returned error can't find the container with id a3acfbf6a85079a60e3d7def6419c2394bef8e039f428e42d1b2fb05ed068295 Jul 07 19:29:51.510202 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.510126 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-688nc"] Jul 07 19:29:51.513228 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.513213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.515888 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.515867 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7dvdb\"" Jul 07 19:29:51.516038 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.515868 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 19:29:51.523844 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.523823 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-688nc"] Jul 07 19:29:51.567858 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.567832 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-zwrxv"] Jul 07 19:29:51.570103 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.570088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.582649 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.582626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-zwrxv"] Jul 07 19:29:51.637076 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.637044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cztb6\" (UniqueName: \"kubernetes.io/projected/b9d19a47-7e0d-4b40-b476-108897bf6bc1-kube-api-access-cztb6\") pod \"payload-pre-processing-7964457696-zwrxv\" (UID: \"b9d19a47-7e0d-4b40-b476-108897bf6bc1\") " pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.637224 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.637095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb9660-36c8-491a-97a3-8e2f3bf1364a-plugins-config-volume\") pod \"payload-processing-58f86c7798-688nc\" (UID: \"d4eb9660-36c8-491a-97a3-8e2f3bf1364a\") " pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.637224 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.637180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr6qb\" (UniqueName: \"kubernetes.io/projected/d4eb9660-36c8-491a-97a3-8e2f3bf1364a-kube-api-access-fr6qb\") pod \"payload-processing-58f86c7798-688nc\" (UID: \"d4eb9660-36c8-491a-97a3-8e2f3bf1364a\") " pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.637224 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.637211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19a47-7e0d-4b40-b476-108897bf6bc1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-zwrxv\" (UID: \"b9d19a47-7e0d-4b40-b476-108897bf6bc1\") " pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.738538 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.738474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb9660-36c8-491a-97a3-8e2f3bf1364a-plugins-config-volume\") pod \"payload-processing-58f86c7798-688nc\" (UID: \"d4eb9660-36c8-491a-97a3-8e2f3bf1364a\") " pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.738714 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.738579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr6qb\" (UniqueName: \"kubernetes.io/projected/d4eb9660-36c8-491a-97a3-8e2f3bf1364a-kube-api-access-fr6qb\") pod \"payload-processing-58f86c7798-688nc\" (UID: \"d4eb9660-36c8-491a-97a3-8e2f3bf1364a\") " pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.738714 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.738609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19a47-7e0d-4b40-b476-108897bf6bc1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-zwrxv\" (UID: \"b9d19a47-7e0d-4b40-b476-108897bf6bc1\") " pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.738714 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.738652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cztb6\" (UniqueName: \"kubernetes.io/projected/b9d19a47-7e0d-4b40-b476-108897bf6bc1-kube-api-access-cztb6\") pod \"payload-pre-processing-7964457696-zwrxv\" (UID: \"b9d19a47-7e0d-4b40-b476-108897bf6bc1\") " pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.739417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.739395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19a47-7e0d-4b40-b476-108897bf6bc1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-zwrxv\" (UID: \"b9d19a47-7e0d-4b40-b476-108897bf6bc1\") " pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.739470 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.739395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb9660-36c8-491a-97a3-8e2f3bf1364a-plugins-config-volume\") pod \"payload-processing-58f86c7798-688nc\" (UID: \"d4eb9660-36c8-491a-97a3-8e2f3bf1364a\") " pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.748379 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.748353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr6qb\" (UniqueName: \"kubernetes.io/projected/d4eb9660-36c8-491a-97a3-8e2f3bf1364a-kube-api-access-fr6qb\") pod \"payload-processing-58f86c7798-688nc\" (UID: \"d4eb9660-36c8-491a-97a3-8e2f3bf1364a\") " pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.748499 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.748398 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cztb6\" (UniqueName: \"kubernetes.io/projected/b9d19a47-7e0d-4b40-b476-108897bf6bc1-kube-api-access-cztb6\") pod \"payload-pre-processing-7964457696-zwrxv\" (UID: \"b9d19a47-7e0d-4b40-b476-108897bf6bc1\") " pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.824385 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.824348 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:51.879823 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.879786 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:51.965814 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:51.965766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-688nc"] Jul 07 19:29:51.969687 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:29:51.969660 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4eb9660_36c8_491a_97a3_8e2f3bf1364a.slice/crio-c89abbac510ee6f7f624f109260199175baf96035a308cf04182d4c5163ccc93 WatchSource:0}: Error finding container c89abbac510ee6f7f624f109260199175baf96035a308cf04182d4c5163ccc93: Status 404 returned error can't find the container with id c89abbac510ee6f7f624f109260199175baf96035a308cf04182d4c5163ccc93 Jul 07 19:29:52.017630 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:52.017605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-zwrxv"] Jul 07 19:29:52.019138 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:29:52.019112 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d19a47_7e0d_4b40_b476_108897bf6bc1.slice/crio-7a4cba55598413c289080f825dfd332698b808f0f546d0b1aee215cee69a23ce WatchSource:0}: Error finding container 7a4cba55598413c289080f825dfd332698b808f0f546d0b1aee215cee69a23ce: Status 404 returned error can't find the container with id 7a4cba55598413c289080f825dfd332698b808f0f546d0b1aee215cee69a23ce Jul 07 19:29:52.371108 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:52.371069 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-688nc" event={"ID":"d4eb9660-36c8-491a-97a3-8e2f3bf1364a","Type":"ContainerStarted","Data":"c89abbac510ee6f7f624f109260199175baf96035a308cf04182d4c5163ccc93"} Jul 07 19:29:52.372394 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:52.372368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" event={"ID":"b9d19a47-7e0d-4b40-b476-108897bf6bc1","Type":"ContainerStarted","Data":"7a4cba55598413c289080f825dfd332698b808f0f546d0b1aee215cee69a23ce"} Jul 07 19:29:52.373514 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:52.373475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6d85ff795b-skvnl" event={"ID":"124bc3ca-de48-4066-9b81-c4c4942e40be","Type":"ContainerStarted","Data":"a3acfbf6a85079a60e3d7def6419c2394bef8e039f428e42d1b2fb05ed068295"} Jul 07 19:29:54.388315 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.388284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" event={"ID":"b9d19a47-7e0d-4b40-b476-108897bf6bc1","Type":"ContainerStarted","Data":"0b4d6f85d8cb0ac91e2e492e6fe422ffc8164d74b28b8359ec76fefa02e541d4"} Jul 07 19:29:54.388742 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.388405 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:29:54.389832 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.389816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6d85ff795b-skvnl" event={"ID":"124bc3ca-de48-4066-9b81-c4c4942e40be","Type":"ContainerStarted","Data":"48f97350f3c13ef24279ba6191acf56641e227edd7ba3d84a606076275e23085"} Jul 07 19:29:54.389912 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.389898 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:29:54.391168 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.391149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-688nc" event={"ID":"d4eb9660-36c8-491a-97a3-8e2f3bf1364a","Type":"ContainerStarted","Data":"a69aea2307979124d552290dee9e5c585ea502b21b06e3c35c0cc01cf69e8326"} Jul 07 19:29:54.391260 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.391247 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:29:54.407141 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.407096 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" podStartSLOduration=1.729969832 podStartE2EDuration="3.407081267s" totalImagesPullingTime="1.677111435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:54.404843991 +0000 UTC m=+510.189829020" watchObservedRunningTime="2026-07-07 19:29:54.407081267 +0000 UTC m=+510.192066266" Jul 07 19:29:54.422765 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.422722 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-688nc" podStartSLOduration=1.699425886 podStartE2EDuration="3.42270753s" totalImagesPullingTime="1.723281644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:54.421318309 +0000 UTC m=+510.206303313" watchObservedRunningTime="2026-07-07 19:29:54.42270753 +0000 UTC m=+510.207692530" Jul 07 19:29:54.442262 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:29:54.442218 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6d85ff795b-skvnl" podStartSLOduration=2.133585221 podStartE2EDuration="4.442203119s" totalImagesPullingTime="2.308617898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:29:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:54.440359996 +0000 UTC m=+510.225345021" watchObservedRunningTime="2026-07-07 19:29:54.442203119 +0000 UTC m=+510.227188147" Jul 07 19:30:00.143820 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.143784 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724210-cph2q"] Jul 07 19:30:00.146664 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.146649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:00.158041 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.158015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724210-cph2q"] Jul 07 19:30:00.312963 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.312925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz9br\" (UniqueName: \"kubernetes.io/projected/d2a7a695-f69f-4769-a8ee-ce1d51701f1d-kube-api-access-fz9br\") pod \"maas-api-key-cleanup-29724210-cph2q\" (UID: \"d2a7a695-f69f-4769-a8ee-ce1d51701f1d\") " pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:00.402046 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.401951 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6d85ff795b-skvnl" Jul 07 19:30:00.413653 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.413626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz9br\" (UniqueName: \"kubernetes.io/projected/d2a7a695-f69f-4769-a8ee-ce1d51701f1d-kube-api-access-fz9br\") pod \"maas-api-key-cleanup-29724210-cph2q\" (UID: \"d2a7a695-f69f-4769-a8ee-ce1d51701f1d\") " pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:00.426131 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.426102 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz9br\" (UniqueName: \"kubernetes.io/projected/d2a7a695-f69f-4769-a8ee-ce1d51701f1d-kube-api-access-fz9br\") pod \"maas-api-key-cleanup-29724210-cph2q\" (UID: \"d2a7a695-f69f-4769-a8ee-ce1d51701f1d\") " pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:00.457043 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.457017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:00.589433 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:00.589198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724210-cph2q"] Jul 07 19:30:00.591763 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:30:00.591736 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a7a695_f69f_4769_a8ee_ce1d51701f1d.slice/crio-1fb7c6ef0a0be320c8b9d80d08fbb5a6c59d998148cce480c1bdbb914efea4ad WatchSource:0}: Error finding container 1fb7c6ef0a0be320c8b9d80d08fbb5a6c59d998148cce480c1bdbb914efea4ad: Status 404 returned error can't find the container with id 1fb7c6ef0a0be320c8b9d80d08fbb5a6c59d998148cce480c1bdbb914efea4ad Jul 07 19:30:01.426550 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:01.426519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" event={"ID":"d2a7a695-f69f-4769-a8ee-ce1d51701f1d","Type":"ContainerStarted","Data":"1fb7c6ef0a0be320c8b9d80d08fbb5a6c59d998148cce480c1bdbb914efea4ad"} Jul 07 19:30:03.446048 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:03.446007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" event={"ID":"d2a7a695-f69f-4769-a8ee-ce1d51701f1d","Type":"ContainerStarted","Data":"66e704d9a1d26513c9f7fd70138e7df0a0253951735ad38b0401e3a3f831f19f"} Jul 07 19:30:03.464286 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:03.464189 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" podStartSLOduration=0.860098801 podStartE2EDuration="3.464174505s" totalImagesPullingTime="2.604075704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:30:03.464120105 +0000 UTC m=+519.249105132" watchObservedRunningTime="2026-07-07 19:30:03.464174505 +0000 UTC m=+519.249159505" Jul 07 19:30:04.453103 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:04.453066 2580 generic.go:356] "Generic (PLEG): container finished" podID="d2a7a695-f69f-4769-a8ee-ce1d51701f1d" containerID="66e704d9a1d26513c9f7fd70138e7df0a0253951735ad38b0401e3a3f831f19f" exitCode=0 Jul 07 19:30:04.453496 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:04.453153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" event={"ID":"d2a7a695-f69f-4769-a8ee-ce1d51701f1d","Type":"ContainerDied","Data":"66e704d9a1d26513c9f7fd70138e7df0a0253951735ad38b0401e3a3f831f19f"} Jul 07 19:30:05.398304 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:05.398277 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-688nc" Jul 07 19:30:05.398509 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:05.398346 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-zwrxv" Jul 07 19:30:05.587895 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:05.587869 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:05.665007 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:05.664890 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz9br\" (UniqueName: \"kubernetes.io/projected/d2a7a695-f69f-4769-a8ee-ce1d51701f1d-kube-api-access-fz9br\") pod \"d2a7a695-f69f-4769-a8ee-ce1d51701f1d\" (UID: \"d2a7a695-f69f-4769-a8ee-ce1d51701f1d\") " Jul 07 19:30:05.667177 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:05.667147 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a7a695-f69f-4769-a8ee-ce1d51701f1d-kube-api-access-fz9br" (OuterVolumeSpecName: "kube-api-access-fz9br") pod "d2a7a695-f69f-4769-a8ee-ce1d51701f1d" (UID: "d2a7a695-f69f-4769-a8ee-ce1d51701f1d"). InnerVolumeSpecName "kube-api-access-fz9br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:30:05.765553 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:05.765510 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz9br\" (UniqueName: \"kubernetes.io/projected/d2a7a695-f69f-4769-a8ee-ce1d51701f1d-kube-api-access-fz9br\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:30:06.464641 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:06.464611 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" Jul 07 19:30:06.464818 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:06.464654 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724210-cph2q" event={"ID":"d2a7a695-f69f-4769-a8ee-ce1d51701f1d","Type":"ContainerDied","Data":"1fb7c6ef0a0be320c8b9d80d08fbb5a6c59d998148cce480c1bdbb914efea4ad"} Jul 07 19:30:06.464818 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:06.464715 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fb7c6ef0a0be320c8b9d80d08fbb5a6c59d998148cce480c1bdbb914efea4ad" Jul 07 19:30:17.092584 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.092549 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66"] Jul 07 19:30:17.092922 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.092876 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a7a695-f69f-4769-a8ee-ce1d51701f1d" containerName="cleanup" Jul 07 19:30:17.092922 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.092886 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a7a695-f69f-4769-a8ee-ce1d51701f1d" containerName="cleanup" Jul 07 19:30:17.093029 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.092950 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a7a695-f69f-4769-a8ee-ce1d51701f1d" containerName="cleanup" Jul 07 19:30:17.094980 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.094954 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.098771 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.098744 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 19:30:17.098884 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.098754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 19:30:17.098884 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.098754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 19:30:17.098884 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.098755 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-962vw\"" Jul 07 19:30:17.113279 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.113254 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66"] Jul 07 19:30:17.141437 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.141402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.141579 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.141449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl8vg\" (UniqueName: \"kubernetes.io/projected/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-kube-api-access-pl8vg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.141579 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.141500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.141579 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.141520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.141579 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.141550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.141704 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.141589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242015 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.241950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242015 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242248 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242248 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242071 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242363 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242363 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl8vg\" (UniqueName: \"kubernetes.io/projected/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-kube-api-access-pl8vg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242442 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.242556 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.242536 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.244409 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.244387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.244567 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.244552 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.252319 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.252298 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl8vg\" (UniqueName: \"kubernetes.io/projected/4ad22d1b-8a06-43a0-8eb1-cedb0099267c-kube-api-access-pl8vg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p4h66\" (UID: \"4ad22d1b-8a06-43a0-8eb1-cedb0099267c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.405816 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.405716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:17.549254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:17.547452 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66"] Jul 07 19:30:18.525491 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:18.525457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" event={"ID":"4ad22d1b-8a06-43a0-8eb1-cedb0099267c","Type":"ContainerStarted","Data":"6a268591bb8d5973664f675ff50fb59f9be0d2fd617de5c26a10b066d4651d2f"} Jul 07 19:30:19.532223 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:19.532190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" event={"ID":"4ad22d1b-8a06-43a0-8eb1-cedb0099267c","Type":"ContainerStarted","Data":"b152e0f9ff5b60a18955b9a3ede319aca2a2c43b7ac6b9b2b9ab6791d8d535bd"} Jul 07 19:30:19.557346 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:19.557295 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" podStartSLOduration=1.5695270080000001 podStartE2EDuration="2.557275845s" totalImagesPullingTime="987.748837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:30:19.555030427 +0000 UTC m=+535.340015428" watchObservedRunningTime="2026-07-07 19:30:19.557275845 +0000 UTC m=+535.342260844" Jul 07 19:30:27.406111 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:27.406051 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:27.406542 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:27.406138 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:27.419117 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:27.419087 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:27.583946 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:27.583919 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p4h66" Jul 07 19:30:40.469282 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.469252 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r"] Jul 07 19:30:40.475645 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.475623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.478091 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.478071 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 19:30:40.486410 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.486387 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r"] Jul 07 19:30:40.539254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.539213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.539254 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.539261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.539459 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.539285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.539459 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.539357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0011fb34-578b-409c-92f2-ff13518520b8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.539459 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.539393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg7zg\" (UniqueName: \"kubernetes.io/projected/0011fb34-578b-409c-92f2-ff13518520b8-kube-api-access-sg7zg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.539459 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.539430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640386 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg7zg\" (UniqueName: \"kubernetes.io/projected/0011fb34-578b-409c-92f2-ff13518520b8-kube-api-access-sg7zg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640613 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640613 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640613 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640613 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640821 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0011fb34-578b-409c-92f2-ff13518520b8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640876 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640816 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.640935 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640883 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.641008 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.640949 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.642778 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.642752 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0011fb34-578b-409c-92f2-ff13518520b8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.643060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.643043 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0011fb34-578b-409c-92f2-ff13518520b8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.649854 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.649828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg7zg\" (UniqueName: \"kubernetes.io/projected/0011fb34-578b-409c-92f2-ff13518520b8-kube-api-access-sg7zg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lz65r\" (UID: \"0011fb34-578b-409c-92f2-ff13518520b8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.786571 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.786483 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:40.920574 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:40.920544 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r"] Jul 07 19:30:40.922515 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:30:40.922482 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0011fb34_578b_409c_92f2_ff13518520b8.slice/crio-264e9e13851652533092a04afa5185dad247daa8837656c9f28a5b1a256756ee WatchSource:0}: Error finding container 264e9e13851652533092a04afa5185dad247daa8837656c9f28a5b1a256756ee: Status 404 returned error can't find the container with id 264e9e13851652533092a04afa5185dad247daa8837656c9f28a5b1a256756ee Jul 07 19:30:41.643231 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:41.643193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" event={"ID":"0011fb34-578b-409c-92f2-ff13518520b8","Type":"ContainerStarted","Data":"a39e999cb7b434534e6309514c71049f9367ec37e7f10097fdf6cb676b13b536"} Jul 07 19:30:41.643608 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:41.643293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" event={"ID":"0011fb34-578b-409c-92f2-ff13518520b8","Type":"ContainerStarted","Data":"264e9e13851652533092a04afa5185dad247daa8837656c9f28a5b1a256756ee"} Jul 07 19:30:41.668338 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:41.668283 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" podStartSLOduration=1.479888533 podStartE2EDuration="1.668265492s" totalImagesPullingTime="188.376959ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:30:41.665503311 +0000 UTC m=+557.450488324" watchObservedRunningTime="2026-07-07 19:30:41.668265492 +0000 UTC m=+557.453250492" Jul 07 19:30:50.786704 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:50.786660 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:50.787157 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:50.786733 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:50.799560 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:50.799536 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:30:51.705275 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:30:51.705248 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lz65r" Jul 07 19:31:06.375261 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.375223 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff"] Jul 07 19:31:06.379240 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.379220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.381739 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.381717 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 19:31:06.391000 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.390952 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff"] Jul 07 19:31:06.463348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.463312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvtql\" (UniqueName: \"kubernetes.io/projected/0edda581-6410-41af-82a0-f501316419e1-kube-api-access-tvtql\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.463348 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.463351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.463571 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.463423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.463571 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.463477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.463571 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.463522 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.463571 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.463555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0edda581-6410-41af-82a0-f501316419e1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.564848 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.564811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0edda581-6410-41af-82a0-f501316419e1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.564882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvtql\" (UniqueName: \"kubernetes.io/projected/0edda581-6410-41af-82a0-f501316419e1-kube-api-access-tvtql\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.564907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.564953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.565014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565060 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.565042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565494 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.565471 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565647 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.565498 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.565647 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.565514 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.567205 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.567185 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0edda581-6410-41af-82a0-f501316419e1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.567517 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.567501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0edda581-6410-41af-82a0-f501316419e1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.576658 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.576630 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvtql\" (UniqueName: \"kubernetes.io/projected/0edda581-6410-41af-82a0-f501316419e1-kube-api-access-tvtql\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zp4ff\" (UID: \"0edda581-6410-41af-82a0-f501316419e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.690964 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.690880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:06.844642 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:06.844599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff"] Jul 07 19:31:06.847571 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:31:06.847544 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0edda581_6410_41af_82a0_f501316419e1.slice/crio-01d159629eeb24d58111c7d84bf83cbe16b437b68a93595cf394d82219b1dfcd WatchSource:0}: Error finding container 01d159629eeb24d58111c7d84bf83cbe16b437b68a93595cf394d82219b1dfcd: Status 404 returned error can't find the container with id 01d159629eeb24d58111c7d84bf83cbe16b437b68a93595cf394d82219b1dfcd Jul 07 19:31:07.783591 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:07.783561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" event={"ID":"0edda581-6410-41af-82a0-f501316419e1","Type":"ContainerStarted","Data":"e3c8338ae29d429a36d75df34f07633e5313994166b76d5aeffd4a00aa4b2dd7"} Jul 07 19:31:07.784027 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:07.783612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" event={"ID":"0edda581-6410-41af-82a0-f501316419e1","Type":"ContainerStarted","Data":"01d159629eeb24d58111c7d84bf83cbe16b437b68a93595cf394d82219b1dfcd"} Jul 07 19:31:07.819419 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:07.819367 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" podStartSLOduration=1.644981909 podStartE2EDuration="1.819351591s" totalImagesPullingTime="174.369682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:31:07.818672978 +0000 UTC m=+583.603657998" watchObservedRunningTime="2026-07-07 19:31:07.819351591 +0000 UTC m=+583.604336640" Jul 07 19:31:16.692031 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:16.691922 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:16.692386 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:16.692037 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:16.704356 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:16.704328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:31:16.845772 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:31:16.845742 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zp4ff" Jul 07 19:33:25.032217 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.032180 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-754f59f895-tj24b"] Jul 07 19:33:25.033078 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.032425 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-754f59f895-tj24b" podUID="4e2b80a5-3083-431e-9618-f750169424f0" containerName="manager" containerID="cri-o://96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177" gracePeriod=30 Jul 07 19:33:25.287660 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.287600 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:33:25.397124 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.397089 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74z6m\" (UniqueName: \"kubernetes.io/projected/4e2b80a5-3083-431e-9618-f750169424f0-kube-api-access-74z6m\") pod \"4e2b80a5-3083-431e-9618-f750169424f0\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " Jul 07 19:33:25.397324 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.397186 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e2b80a5-3083-431e-9618-f750169424f0-webhook-cert\") pod \"4e2b80a5-3083-431e-9618-f750169424f0\" (UID: \"4e2b80a5-3083-431e-9618-f750169424f0\") " Jul 07 19:33:25.399318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.399283 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e2b80a5-3083-431e-9618-f750169424f0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4e2b80a5-3083-431e-9618-f750169424f0" (UID: "4e2b80a5-3083-431e-9618-f750169424f0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:33:25.399419 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.399353 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2b80a5-3083-431e-9618-f750169424f0-kube-api-access-74z6m" (OuterVolumeSpecName: "kube-api-access-74z6m") pod "4e2b80a5-3083-431e-9618-f750169424f0" (UID: "4e2b80a5-3083-431e-9618-f750169424f0"). InnerVolumeSpecName "kube-api-access-74z6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:33:25.496834 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.496805 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e2b80a5-3083-431e-9618-f750169424f0" containerID="96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177" exitCode=0 Jul 07 19:33:25.497001 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.496847 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754f59f895-tj24b" Jul 07 19:33:25.497001 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.496858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754f59f895-tj24b" event={"ID":"4e2b80a5-3083-431e-9618-f750169424f0","Type":"ContainerDied","Data":"96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177"} Jul 07 19:33:25.497001 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.496899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754f59f895-tj24b" event={"ID":"4e2b80a5-3083-431e-9618-f750169424f0","Type":"ContainerDied","Data":"21a5073747ab4d7b2cb576297fd86a7378595a3cee5b49518718f9a154404fd7"} Jul 07 19:33:25.497001 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.496922 2580 scope.go:117] "RemoveContainer" containerID="96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177" Jul 07 19:33:25.497919 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.497898 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74z6m\" (UniqueName: \"kubernetes.io/projected/4e2b80a5-3083-431e-9618-f750169424f0-kube-api-access-74z6m\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:33:25.497919 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.497919 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e2b80a5-3083-431e-9618-f750169424f0-webhook-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:33:25.508658 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.508639 2580 scope.go:117] "RemoveContainer" containerID="96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177" Jul 07 19:33:25.508921 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:33:25.508903 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177\": container with ID starting with 96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177 not found: ID does not exist" containerID="96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177" Jul 07 19:33:25.509054 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.508930 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177"} err="failed to get container status \"96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177\": rpc error: code = NotFound desc = could not find container \"96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177\": container with ID starting with 96316567a2b9148c9ba44cfba24f33fa3b7d7e9076cf85de14265fa935749177 not found: ID does not exist" Jul 07 19:33:25.523526 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.523500 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-754f59f895-tj24b"] Jul 07 19:33:25.529587 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:25.529566 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-754f59f895-tj24b"] Jul 07 19:33:26.306882 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.306849 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-754f59f895-ncpms"] Jul 07 19:33:26.307266 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.307196 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2b80a5-3083-431e-9618-f750169424f0" containerName="manager" Jul 07 19:33:26.307266 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.307208 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2b80a5-3083-431e-9618-f750169424f0" containerName="manager" Jul 07 19:33:26.307266 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.307267 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e2b80a5-3083-431e-9618-f750169424f0" containerName="manager" Jul 07 19:33:26.310228 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.310211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.312649 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.312624 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 19:33:26.312787 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.312758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 19:33:26.313753 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.313735 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wrh4h\"" Jul 07 19:33:26.320487 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.320464 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-754f59f895-ncpms"] Jul 07 19:33:26.506386 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.506355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m7gb\" (UniqueName: \"kubernetes.io/projected/017befe0-911d-42eb-88ad-7068367e757b-kube-api-access-7m7gb\") pod \"maas-controller-754f59f895-ncpms\" (UID: \"017befe0-911d-42eb-88ad-7068367e757b\") " pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.506563 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.506391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/017befe0-911d-42eb-88ad-7068367e757b-webhook-cert\") pod \"maas-controller-754f59f895-ncpms\" (UID: \"017befe0-911d-42eb-88ad-7068367e757b\") " pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.607318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.607217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m7gb\" (UniqueName: \"kubernetes.io/projected/017befe0-911d-42eb-88ad-7068367e757b-kube-api-access-7m7gb\") pod \"maas-controller-754f59f895-ncpms\" (UID: \"017befe0-911d-42eb-88ad-7068367e757b\") " pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.607318 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.607271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/017befe0-911d-42eb-88ad-7068367e757b-webhook-cert\") pod \"maas-controller-754f59f895-ncpms\" (UID: \"017befe0-911d-42eb-88ad-7068367e757b\") " pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.609674 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.609650 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/017befe0-911d-42eb-88ad-7068367e757b-webhook-cert\") pod \"maas-controller-754f59f895-ncpms\" (UID: \"017befe0-911d-42eb-88ad-7068367e757b\") " pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.616252 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.616233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m7gb\" (UniqueName: \"kubernetes.io/projected/017befe0-911d-42eb-88ad-7068367e757b-kube-api-access-7m7gb\") pod \"maas-controller-754f59f895-ncpms\" (UID: \"017befe0-911d-42eb-88ad-7068367e757b\") " pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.622045 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.622025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:26.757019 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.756989 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-754f59f895-ncpms"] Jul 07 19:33:26.757897 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:33:26.757872 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017befe0_911d_42eb_88ad_7068367e757b.slice/crio-7483751574aaece3cd78e64b273d216dac73c0f1145e9e7d42c255f59edb0463 WatchSource:0}: Error finding container 7483751574aaece3cd78e64b273d216dac73c0f1145e9e7d42c255f59edb0463: Status 404 returned error can't find the container with id 7483751574aaece3cd78e64b273d216dac73c0f1145e9e7d42c255f59edb0463 Jul 07 19:33:26.759221 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.759202 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:33:26.916823 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:26.916745 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e2b80a5-3083-431e-9618-f750169424f0" path="/var/lib/kubelet/pods/4e2b80a5-3083-431e-9618-f750169424f0/volumes" Jul 07 19:33:27.510475 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:27.510379 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754f59f895-ncpms" event={"ID":"017befe0-911d-42eb-88ad-7068367e757b","Type":"ContainerStarted","Data":"6a7facd511dac9c3f882458c0a01373f4decd6698d836aac039310611a9bc2a4"} Jul 07 19:33:27.510475 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:27.510439 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:33:27.510475 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:27.510456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754f59f895-ncpms" event={"ID":"017befe0-911d-42eb-88ad-7068367e757b","Type":"ContainerStarted","Data":"7483751574aaece3cd78e64b273d216dac73c0f1145e9e7d42c255f59edb0463"} Jul 07 19:33:27.530533 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:27.530481 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-754f59f895-ncpms" podStartSLOduration=1.120170107 podStartE2EDuration="1.530463188s" totalImagesPullingTime="410.293081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:33:27.528934992 +0000 UTC m=+723.313919996" watchObservedRunningTime="2026-07-07 19:33:27.530463188 +0000 UTC m=+723.315448188" Jul 07 19:33:38.522798 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:33:38.522768 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-754f59f895-ncpms" Jul 07 19:47:59.408741 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.408703 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dd8895475-dm8m4"] Jul 07 19:47:59.411984 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.411958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.422415 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.422389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dd8895475-dm8m4"] Jul 07 19:47:59.435235 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-console-config\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.435349 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1500a24e-cd81-44e1-ade6-05c6d716d032-console-serving-cert\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.435349 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-oauth-serving-cert\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.435349 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4zk7\" (UniqueName: \"kubernetes.io/projected/1500a24e-cd81-44e1-ade6-05c6d716d032-kube-api-access-t4zk7\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.435349 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1500a24e-cd81-44e1-ade6-05c6d716d032-console-oauth-config\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.435507 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-service-ca\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.435507 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.435405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-trusted-ca-bundle\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.535855 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.535819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-console-config\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.535855 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.535856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1500a24e-cd81-44e1-ade6-05c6d716d032-console-serving-cert\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536098 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.535875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-oauth-serving-cert\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536098 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.535896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4zk7\" (UniqueName: \"kubernetes.io/projected/1500a24e-cd81-44e1-ade6-05c6d716d032-kube-api-access-t4zk7\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536098 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.535951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1500a24e-cd81-44e1-ade6-05c6d716d032-console-oauth-config\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536098 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.536004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-service-ca\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536280 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.536207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-trusted-ca-bundle\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536682 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.536659 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-service-ca\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536793 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.536662 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-console-config\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.536793 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.536663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-oauth-serving-cert\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.537120 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.537099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1500a24e-cd81-44e1-ade6-05c6d716d032-trusted-ca-bundle\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.538393 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.538376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1500a24e-cd81-44e1-ade6-05c6d716d032-console-oauth-config\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.538545 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.538530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1500a24e-cd81-44e1-ade6-05c6d716d032-console-serving-cert\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.546044 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.546022 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4zk7\" (UniqueName: \"kubernetes.io/projected/1500a24e-cd81-44e1-ade6-05c6d716d032-kube-api-access-t4zk7\") pod \"console-5dd8895475-dm8m4\" (UID: \"1500a24e-cd81-44e1-ade6-05c6d716d032\") " pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.721887 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.721810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:47:59.858448 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.858422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dd8895475-dm8m4"] Jul 07 19:47:59.860182 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:47:59.860153 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1500a24e_cd81_44e1_ade6_05c6d716d032.slice/crio-28dfb8636687ec1922e7168d35ee74805bec3d957c27ef5c02b48ff80805f2f6 WatchSource:0}: Error finding container 28dfb8636687ec1922e7168d35ee74805bec3d957c27ef5c02b48ff80805f2f6: Status 404 returned error can't find the container with id 28dfb8636687ec1922e7168d35ee74805bec3d957c27ef5c02b48ff80805f2f6 Jul 07 19:47:59.977777 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.977684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dd8895475-dm8m4" event={"ID":"1500a24e-cd81-44e1-ade6-05c6d716d032","Type":"ContainerStarted","Data":"bc2152fa4eab59df5b179ea87019c5f6e2840cb99ede44ee9d404b6dfcb58342"} Jul 07 19:47:59.977777 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.977739 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dd8895475-dm8m4" event={"ID":"1500a24e-cd81-44e1-ade6-05c6d716d032","Type":"ContainerStarted","Data":"28dfb8636687ec1922e7168d35ee74805bec3d957c27ef5c02b48ff80805f2f6"} Jul 07 19:47:59.997056 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:47:59.997005 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dd8895475-dm8m4" podStartSLOduration=0.99698649 podStartE2EDuration="996.98649ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:47:59.995731823 +0000 UTC m=+1595.780716824" watchObservedRunningTime="2026-07-07 19:47:59.99698649 +0000 UTC m=+1595.781971487" Jul 07 19:48:09.722056 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:09.722021 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:48:09.722452 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:09.722077 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:48:09.726679 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:09.726658 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:48:10.039200 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:10.039119 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dd8895475-dm8m4" Jul 07 19:48:10.121456 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:10.121422 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d7995947-qbw2c"] Jul 07 19:48:35.141826 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.141719 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d7995947-qbw2c" podUID="56c6653c-28f6-4daf-ba2e-7de689b1cd78" containerName="console" containerID="cri-o://6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3" gracePeriod=15 Jul 07 19:48:35.389576 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.389552 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:48:35.554940 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.554914 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8j59\" (UniqueName: \"kubernetes.io/projected/56c6653c-28f6-4daf-ba2e-7de689b1cd78-kube-api-access-k8j59\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555106 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.554957 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-oauth-config\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555106 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555076 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-service-ca\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555195 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555117 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-serving-cert\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555195 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555159 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-trusted-ca-bundle\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555195 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555183 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-oauth-serving-cert\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555342 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555210 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-config\") pod \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\" (UID: \"56c6653c-28f6-4daf-ba2e-7de689b1cd78\") " Jul 07 19:48:35.555515 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555469 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-service-ca" (OuterVolumeSpecName: "service-ca") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:48:35.555723 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555535 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:48:35.555723 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555542 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:48:35.555838 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.555758 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-config" (OuterVolumeSpecName: "console-config") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:48:35.557285 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.557262 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56c6653c-28f6-4daf-ba2e-7de689b1cd78-kube-api-access-k8j59" (OuterVolumeSpecName: "kube-api-access-k8j59") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "kube-api-access-k8j59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:48:35.557389 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.557261 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:48:35.557389 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.557285 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "56c6653c-28f6-4daf-ba2e-7de689b1cd78" (UID: "56c6653c-28f6-4daf-ba2e-7de689b1cd78"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:48:35.656580 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656548 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-trusted-ca-bundle\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:35.656580 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656573 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-oauth-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:35.656580 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656584 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:35.656794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656595 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8j59\" (UniqueName: \"kubernetes.io/projected/56c6653c-28f6-4daf-ba2e-7de689b1cd78-kube-api-access-k8j59\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:35.656794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656605 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-oauth-config\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:35.656794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656613 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56c6653c-28f6-4daf-ba2e-7de689b1cd78-service-ca\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:35.656794 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:35.656623 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56c6653c-28f6-4daf-ba2e-7de689b1cd78-console-serving-cert\") on node \"ip-10-0-135-242.ec2.internal\" DevicePath \"\"" Jul 07 19:48:36.172471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.172437 2580 generic.go:356] "Generic (PLEG): container finished" podID="56c6653c-28f6-4daf-ba2e-7de689b1cd78" containerID="6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3" exitCode=0 Jul 07 19:48:36.172916 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.172485 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7995947-qbw2c" Jul 07 19:48:36.172916 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.172534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7995947-qbw2c" event={"ID":"56c6653c-28f6-4daf-ba2e-7de689b1cd78","Type":"ContainerDied","Data":"6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3"} Jul 07 19:48:36.172916 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.172590 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7995947-qbw2c" event={"ID":"56c6653c-28f6-4daf-ba2e-7de689b1cd78","Type":"ContainerDied","Data":"b88a657779a55b66aa5f2470db41d782119358fb260ef56fa532fbbabd83bfbc"} Jul 07 19:48:36.172916 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.172621 2580 scope.go:117] "RemoveContainer" containerID="6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3" Jul 07 19:48:36.181444 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.181428 2580 scope.go:117] "RemoveContainer" containerID="6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3" Jul 07 19:48:36.181707 ip-10-0-135-242 kubenswrapper[2580]: E0707 19:48:36.181689 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3\": container with ID starting with 6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3 not found: ID does not exist" containerID="6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3" Jul 07 19:48:36.181765 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.181720 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3"} err="failed to get container status \"6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3\": rpc error: code = NotFound desc = could not find container \"6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3\": container with ID starting with 6ba94c781a8d18145f41563bded64beb339db116cfd7916c9d4080d7740db3b3 not found: ID does not exist" Jul 07 19:48:36.197366 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.197338 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d7995947-qbw2c"] Jul 07 19:48:36.202235 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.202213 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d7995947-qbw2c"] Jul 07 19:48:36.916283 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:48:36.916253 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56c6653c-28f6-4daf-ba2e-7de689b1cd78" path="/var/lib/kubelet/pods/56c6653c-28f6-4daf-ba2e-7de689b1cd78/volumes" Jul 07 19:49:39.517917 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:49:39.517877 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-754f59f895-ncpms" podUID="017befe0-911d-42eb-88ad-7068367e757b" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.44:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 19:59:08.875359 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:08.875329 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6d85ff795b-skvnl_124bc3ca-de48-4066-9b81-c4c4942e40be/maas-api/0.log" Jul 07 19:59:08.991409 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:08.991379 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724210-cph2q_d2a7a695-f69f-4769-a8ee-ce1d51701f1d/cleanup/0.log" Jul 07 19:59:09.222615 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:09.222497 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-754f59f895-ncpms_017befe0-911d-42eb-88ad-7068367e757b/manager/0.log" Jul 07 19:59:09.460832 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:09.460785 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-w2cqz_ed0984e8-4daf-49c8-b7d9-aff614349488/manager/0.log" Jul 07 19:59:09.575080 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:09.575060 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-jtqvh_1f7143ad-8fbc-4179-a29e-de11b7d40c1b/copy-manifests/0.log" Jul 07 19:59:09.587690 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:09.587665 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-jtqvh_1f7143ad-8fbc-4179-a29e-de11b7d40c1b/manager/0.log" Jul 07 19:59:09.979134 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:09.979047 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-bcq9d_d5ccdece-a292-4aa6-810d-9364f00ef4d8/postgres/0.log" Jul 07 19:59:10.743068 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:10.743033 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59_c0f2e46a-3445-434a-8fc4-15c4e731a881/util/0.log" Jul 07 19:59:10.751912 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:10.751885 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59_c0f2e46a-3445-434a-8fc4-15c4e731a881/pull/0.log" Jul 07 19:59:10.762743 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:10.762712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59_c0f2e46a-3445-434a-8fc4-15c4e731a881/extract/0.log" Jul 07 19:59:10.882216 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:10.882185 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8_8d0d2248-2220-4a1d-8166-ac5dcc5e45a6/util/0.log" Jul 07 19:59:10.890277 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:10.890252 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8_8d0d2248-2220-4a1d-8166-ac5dcc5e45a6/pull/0.log" Jul 07 19:59:10.898541 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:10.898512 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8_8d0d2248-2220-4a1d-8166-ac5dcc5e45a6/extract/0.log" Jul 07 19:59:11.006551 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:11.006479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl_c20f4295-f762-4eb4-a793-bcff05004ede/util/0.log" Jul 07 19:59:11.014360 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:11.014336 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl_c20f4295-f762-4eb4-a793-bcff05004ede/pull/0.log" Jul 07 19:59:11.022786 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:11.022766 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl_c20f4295-f762-4eb4-a793-bcff05004ede/extract/0.log" Jul 07 19:59:11.139547 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:11.139520 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj_bc7699ac-a26e-4a0a-8511-136482e94baf/util/0.log" Jul 07 19:59:11.147623 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:11.147604 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj_bc7699ac-a26e-4a0a-8511-136482e94baf/pull/0.log" Jul 07 19:59:11.156102 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:11.156086 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj_bc7699ac-a26e-4a0a-8511-136482e94baf/extract/0.log" Jul 07 19:59:12.650212 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:12.650181 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bbzm_992194d3-c7d6-4a92-8643-5a84c4467664/discovery/0.log" Jul 07 19:59:13.128845 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:13.128814 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zwrxv_b9d19a47-7e0d-4b40-b476-108897bf6bc1/payload-pre-processing/0.log" Jul 07 19:59:13.260180 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:13.260153 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-688nc_d4eb9660-36c8-491a-97a3-8e2f3bf1364a/payload-processing/0.log" Jul 07 19:59:13.844862 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:13.844827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-p4h66_4ad22d1b-8a06-43a0-8eb1-cedb0099267c/main/0.log" Jul 07 19:59:13.960267 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:13.960244 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lz65r_0011fb34-578b-409c-92f2-ff13518520b8/main/0.log" Jul 07 19:59:14.193156 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:14.193084 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-zp4ff_0edda581-6410-41af-82a0-f501316419e1/main/0.log" Jul 07 19:59:22.040128 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:22.040097 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gs7hm_6be412fb-041b-44dc-b9a9-fbd8792bfd19/global-pull-secret-syncer/0.log" Jul 07 19:59:22.241775 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:22.241746 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j8cqf_e20f31af-c455-4ddf-92ad-db1374f312e1/konnectivity-agent/0.log" Jul 07 19:59:22.355503 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:22.355475 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-242.ec2.internal_b89d234948b15c1959046b4ec18ba250/haproxy/0.log" Jul 07 19:59:26.361593 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.361560 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59_c0f2e46a-3445-434a-8fc4-15c4e731a881/extract/0.log" Jul 07 19:59:26.396181 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.396152 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59_c0f2e46a-3445-434a-8fc4-15c4e731a881/util/0.log" Jul 07 19:59:26.432388 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.432361 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597wl59_c0f2e46a-3445-434a-8fc4-15c4e731a881/pull/0.log" Jul 07 19:59:26.493556 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.493532 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8_8d0d2248-2220-4a1d-8166-ac5dcc5e45a6/extract/0.log" Jul 07 19:59:26.518253 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.518234 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8_8d0d2248-2220-4a1d-8166-ac5dcc5e45a6/util/0.log" Jul 07 19:59:26.544758 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.544724 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08l4q8_8d0d2248-2220-4a1d-8166-ac5dcc5e45a6/pull/0.log" Jul 07 19:59:26.577513 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.577477 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl_c20f4295-f762-4eb4-a793-bcff05004ede/extract/0.log" Jul 07 19:59:26.613626 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.613553 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl_c20f4295-f762-4eb4-a793-bcff05004ede/util/0.log" Jul 07 19:59:26.639768 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.639739 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tv7hl_c20f4295-f762-4eb4-a793-bcff05004ede/pull/0.log" Jul 07 19:59:26.674962 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.674938 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj_bc7699ac-a26e-4a0a-8511-136482e94baf/extract/0.log" Jul 07 19:59:26.706376 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.706338 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj_bc7699ac-a26e-4a0a-8511-136482e94baf/util/0.log" Jul 07 19:59:26.735170 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:26.735134 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nx4xj_bc7699ac-a26e-4a0a-8511-136482e94baf/pull/0.log" Jul 07 19:59:29.700221 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:29.700175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-qvqs9_286e1d46-4c13-4955-a6a7-bd623500eedd/monitoring-plugin/0.log" Jul 07 19:59:29.932778 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:29.932744 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rk8v4_52aa9653-287d-4042-a1b3-49ce179e85e8/node-exporter/0.log" Jul 07 19:59:29.961107 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:29.961027 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rk8v4_52aa9653-287d-4042-a1b3-49ce179e85e8/kube-rbac-proxy/0.log" Jul 07 19:59:29.990694 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:29.990671 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rk8v4_52aa9653-287d-4042-a1b3-49ce179e85e8/init-textfile/0.log" Jul 07 19:59:30.408097 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.408070 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-g28ws_1fe63467-dcd8-4210-bcf5-8da024890cbe/prometheus-operator/0.log" Jul 07 19:59:30.444708 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.444677 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-g28ws_1fe63467-dcd8-4210-bcf5-8da024890cbe/kube-rbac-proxy/0.log" Jul 07 19:59:30.600465 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.600423 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c495498f6-wc5kj_2cab8f4d-b087-434f-a4c5-41afd5161c63/thanos-query/0.log" Jul 07 19:59:30.627097 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.627075 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c495498f6-wc5kj_2cab8f4d-b087-434f-a4c5-41afd5161c63/kube-rbac-proxy-web/0.log" Jul 07 19:59:30.653100 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.653072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c495498f6-wc5kj_2cab8f4d-b087-434f-a4c5-41afd5161c63/kube-rbac-proxy/0.log" Jul 07 19:59:30.680756 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.680694 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c495498f6-wc5kj_2cab8f4d-b087-434f-a4c5-41afd5161c63/prom-label-proxy/0.log" Jul 07 19:59:30.705182 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.705154 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c495498f6-wc5kj_2cab8f4d-b087-434f-a4c5-41afd5161c63/kube-rbac-proxy-rules/0.log" Jul 07 19:59:30.739422 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:30.739400 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c495498f6-wc5kj_2cab8f4d-b087-434f-a4c5-41afd5161c63/kube-rbac-proxy-metrics/0.log" Jul 07 19:59:31.852105 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.852071 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs"] Jul 07 19:59:31.852493 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.852427 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56c6653c-28f6-4daf-ba2e-7de689b1cd78" containerName="console" Jul 07 19:59:31.852493 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.852438 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56c6653c-28f6-4daf-ba2e-7de689b1cd78" containerName="console" Jul 07 19:59:31.852493 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.852488 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="56c6653c-28f6-4daf-ba2e-7de689b1cd78" containerName="console" Jul 07 19:59:31.855634 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.855542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:31.858033 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.858014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x8rrx\"/\"default-dockercfg-65z4s\"" Jul 07 19:59:31.858458 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.858439 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8rrx\"/\"openshift-service-ca.crt\"" Jul 07 19:59:31.860319 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.860303 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8rrx\"/\"kube-root-ca.crt\"" Jul 07 19:59:31.876821 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.876798 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs"] Jul 07 19:59:31.913657 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.913623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-sys\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:31.913657 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.913658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-podres\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:31.913833 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.913692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8fhh\" (UniqueName: \"kubernetes.io/projected/c048d0e2-0c9a-4810-a103-83fb061935b1-kube-api-access-c8fhh\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:31.913833 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.913789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-proc\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:31.913833 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:31.913827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-lib-modules\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015095 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-proc\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015095 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-lib-modules\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-sys\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-podres\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-proc\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8fhh\" (UniqueName: \"kubernetes.io/projected/c048d0e2-0c9a-4810-a103-83fb061935b1-kube-api-access-c8fhh\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015224 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-sys\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-lib-modules\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.015333 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.015302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c048d0e2-0c9a-4810-a103-83fb061935b1-podres\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.024558 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.024531 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8fhh\" (UniqueName: \"kubernetes.io/projected/c048d0e2-0c9a-4810-a103-83fb061935b1-kube-api-access-c8fhh\") pod \"perf-node-gather-daemonset-hggxs\" (UID: \"c048d0e2-0c9a-4810-a103-83fb061935b1\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.165741 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.165658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.297962 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.297928 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs"] Jul 07 19:59:32.299033 ip-10-0-135-242 kubenswrapper[2580]: W0707 19:59:32.299002 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc048d0e2_0c9a_4810_a103_83fb061935b1.slice/crio-d079a3f6610641cb41c7851d53e6c04d8466100621fedf33749d09e46ea24dd2 WatchSource:0}: Error finding container d079a3f6610641cb41c7851d53e6c04d8466100621fedf33749d09e46ea24dd2: Status 404 returned error can't find the container with id d079a3f6610641cb41c7851d53e6c04d8466100621fedf33749d09e46ea24dd2 Jul 07 19:59:32.562417 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.562379 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" event={"ID":"c048d0e2-0c9a-4810-a103-83fb061935b1","Type":"ContainerStarted","Data":"1d7856819c86e54d3a66491d5b85ae5b1da5261b3d46f82c362f5d3393c316e2"} Jul 07 19:59:32.562589 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.562434 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" event={"ID":"c048d0e2-0c9a-4810-a103-83fb061935b1","Type":"ContainerStarted","Data":"d079a3f6610641cb41c7851d53e6c04d8466100621fedf33749d09e46ea24dd2"} Jul 07 19:59:32.562589 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.562492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:32.587265 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:32.587214 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" podStartSLOduration=1.5871995939999999 podStartE2EDuration="1.587199594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:59:32.584667466 +0000 UTC m=+2288.369652465" watchObservedRunningTime="2026-07-07 19:59:32.587199594 +0000 UTC m=+2288.372184595" Jul 07 19:59:33.239775 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:33.239743 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5dd8895475-dm8m4_1500a24e-cd81-44e1-ade6-05c6d716d032/console/0.log" Jul 07 19:59:35.032182 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:35.032158 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l6tpb_6b4866e6-d381-44d0-9238-448b1d785623/dns/0.log" Jul 07 19:59:35.057370 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:35.057349 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l6tpb_6b4866e6-d381-44d0-9238-448b1d785623/kube-rbac-proxy/0.log" Jul 07 19:59:35.239257 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:35.239229 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vjbsv_574fea1d-1d1e-4237-a70d-47f8350db209/dns-node-resolver/0.log" Jul 07 19:59:35.972787 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:35.972748 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zgvz5_7cbfa312-3399-4119-9e04-accaaf744ab6/node-ca/0.log" Jul 07 19:59:37.100957 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:37.100925 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bbzm_992194d3-c7d6-4a92-8643-5a84c4467664/discovery/0.log" Jul 07 19:59:37.270896 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:37.270860 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zwrxv_b9d19a47-7e0d-4b40-b476-108897bf6bc1/payload-pre-processing/0.log" Jul 07 19:59:37.348583 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:37.348557 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-688nc_d4eb9660-36c8-491a-97a3-8e2f3bf1364a/payload-processing/0.log" Jul 07 19:59:38.132400 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:38.132359 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqtsn_489936c9-f207-48f5-9a80-a171c4be51aa/serve-healthcheck-canary/0.log" Jul 07 19:59:38.577351 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:38.577322 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-hggxs" Jul 07 19:59:38.684240 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:38.684212 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m9dvd_0f5670ee-b31c-4476-89b5-f8dac251fb24/kube-rbac-proxy/0.log" Jul 07 19:59:38.713394 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:38.713365 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m9dvd_0f5670ee-b31c-4476-89b5-f8dac251fb24/exporter/0.log" Jul 07 19:59:38.743894 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:38.743869 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m9dvd_0f5670ee-b31c-4476-89b5-f8dac251fb24/extractor/0.log" Jul 07 19:59:41.692500 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:41.692463 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6d85ff795b-skvnl_124bc3ca-de48-4066-9b81-c4c4942e40be/maas-api/0.log" Jul 07 19:59:41.724430 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:41.724405 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724210-cph2q_d2a7a695-f69f-4769-a8ee-ce1d51701f1d/cleanup/0.log" Jul 07 19:59:41.885292 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:41.885259 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-754f59f895-ncpms_017befe0-911d-42eb-88ad-7068367e757b/manager/0.log" Jul 07 19:59:42.002170 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.001881 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-w2cqz_ed0984e8-4daf-49c8-b7d9-aff614349488/manager/0.log" Jul 07 19:59:42.030863 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.030837 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-jtqvh_1f7143ad-8fbc-4179-a29e-de11b7d40c1b/manager/0.log" Jul 07 19:59:42.057901 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.057877 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-jtqvh_1f7143ad-8fbc-4179-a29e-de11b7d40c1b/copy-manifests/0.log" Jul 07 19:59:42.285111 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.285020 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-bcq9d_d5ccdece-a292-4aa6-810d-9364f00ef4d8/postgres/0.log" Jul 07 19:59:42.299894 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.299863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bbzm_992194d3-c7d6-4a92-8643-5a84c4467664/discovery/0.log" Jul 07 19:59:42.423471 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.423442 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zwrxv_b9d19a47-7e0d-4b40-b476-108897bf6bc1/payload-pre-processing/0.log" Jul 07 19:59:42.484754 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:42.484710 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-688nc_d4eb9660-36c8-491a-97a3-8e2f3bf1364a/payload-processing/0.log" Jul 07 19:59:43.699492 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:43.699443 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57f454c8fb-hm2kb_890176dc-a40d-4d9d-9353-5e28aa1fc083/manager/0.log" Jul 07 19:59:43.749697 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:43.749667 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zkcgk_935503c4-91a9-48e9-b937-da41de0b40f8/openshift-lws-operator/0.log" Jul 07 19:59:43.783214 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:43.783107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bbzm_992194d3-c7d6-4a92-8643-5a84c4467664/discovery/0.log" Jul 07 19:59:43.916473 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:43.916442 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zwrxv_b9d19a47-7e0d-4b40-b476-108897bf6bc1/payload-pre-processing/0.log" Jul 07 19:59:43.978249 ip-10-0-135-242 kubenswrapper[2580]: I0707 19:59:43.978219 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-688nc_d4eb9660-36c8-491a-97a3-8e2f3bf1364a/payload-processing/0.log"