Jul 10 13:10:20.160335 ip-10-0-133-196 systemd[1]: Starting Kubernetes Kubelet... Jul 10 13:10:21.220883 ip-10-0-133-196 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 13:10:21.220883 ip-10-0-133-196 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 13:10:21.220883 ip-10-0-133-196 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 13:10:21.220883 ip-10-0-133-196 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 13:10:21.220883 ip-10-0-133-196 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 13:10:21.221715 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.221587 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 13:10:21.228546 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228529 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 13:10:21.228546 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228545 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 13:10:21.228546 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228549 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228553 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228556 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228559 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228562 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228565 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228570 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228575 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228578 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228587 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228591 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228594 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228597 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228600 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228604 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228607 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228611 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228614 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228617 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228620 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 13:10:21.228646 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228623 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228625 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228628 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228631 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228634 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228637 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228639 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228642 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228645 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228648 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228651 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228654 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228657 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228660 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228663 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228666 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228669 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228672 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228675 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228678 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 13:10:21.229147 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228680 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228683 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228686 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228689 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228693 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228697 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228700 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228704 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228707 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228710 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228713 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228717 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228720 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228724 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228728 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228731 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228734 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228737 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228740 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 13:10:21.229659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228742 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228745 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228748 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228751 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228753 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228756 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228759 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228762 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228764 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228767 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228771 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228774 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228777 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228780 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228783 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228785 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228788 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228791 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228794 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228797 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 13:10:21.230151 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228799 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228802 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228806 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228809 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228812 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228814 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228817 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228820 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228823 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228826 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228828 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228831 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228834 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228837 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228840 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228842 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228845 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.228848 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230369 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230378 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 13:10:21.230659 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230382 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230385 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230389 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230392 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230396 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230399 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230402 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230405 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230408 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230410 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230413 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230416 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230419 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230422 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230425 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230428 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230431 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230434 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230437 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230440 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 13:10:21.231174 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230443 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230445 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230449 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230452 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230454 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230457 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230460 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230463 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230468 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230472 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230475 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230477 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230480 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230483 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230486 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230488 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230491 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230494 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230497 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230500 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 13:10:21.231703 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230503 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230505 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230508 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230511 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230514 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230517 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230519 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230522 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230526 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230529 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230533 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230536 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230539 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230542 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230545 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230547 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230550 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230552 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230555 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230558 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 13:10:21.232217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230560 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230563 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230566 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230569 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230571 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230574 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230577 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230581 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230584 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230588 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230591 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230594 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230596 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230600 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230603 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230606 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230608 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230611 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230614 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 13:10:21.232721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230616 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230619 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230621 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230624 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230627 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230629 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230632 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230635 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230638 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230641 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230643 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230646 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230648 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230651 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230654 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230659 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230662 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.230665 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231067 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 13:10:21.233242 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231074 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231077 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231080 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231083 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231087 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231090 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231093 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231095 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231098 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231100 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231103 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231106 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231109 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231112 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231115 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231117 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231120 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231122 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231125 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231147 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 13:10:21.233721 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231152 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231155 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231158 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231161 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231172 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231174 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231177 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231180 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231184 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231188 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231192 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231197 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231200 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231204 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231208 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231212 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231215 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231224 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231227 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 13:10:21.234259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231230 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231233 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231236 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231239 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231242 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231245 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231248 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231251 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231254 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231258 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231262 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231265 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231268 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231271 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231274 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231277 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231280 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231283 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231286 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 13:10:21.234741 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231289 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231292 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231295 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231297 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231301 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231304 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231306 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231309 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231312 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231314 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231317 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231319 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231330 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231333 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231335 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231338 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231341 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231343 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231346 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 13:10:21.235269 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231349 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231351 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231354 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231356 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231359 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231361 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231364 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231367 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231369 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231372 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231375 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231378 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231381 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231384 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231386 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231389 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231392 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231395 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231397 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231400 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.231403 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 13:10:21.235745 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231500 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231512 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231519 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231523 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231528 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231539 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231543 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231548 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231552 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231555 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231559 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231562 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231565 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231569 2571 flags.go:64] FLAG: --cgroup-root="" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231572 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231575 2571 flags.go:64] FLAG: --client-ca-file="" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231578 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231581 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231589 2571 flags.go:64] FLAG: --cluster-domain="" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231592 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231595 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231598 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231602 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231606 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 13:10:21.236298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231609 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231612 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231616 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231619 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231622 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231625 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231629 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231632 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231637 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231640 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231643 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231646 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231650 2571 flags.go:64] FLAG: --enable-server="true" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231653 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231676 2571 flags.go:64] FLAG: --event-burst="100" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231679 2571 flags.go:64] FLAG: --event-qps="50" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231682 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231686 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231689 2571 flags.go:64] FLAG: --eviction-hard="" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231697 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231700 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231703 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231706 2571 flags.go:64] FLAG: --eviction-soft="" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231710 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231713 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 13:10:21.236944 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231716 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231719 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231722 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231725 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231731 2571 flags.go:64] FLAG: --feature-gates="" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231735 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231738 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231741 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231745 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231748 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231751 2571 flags.go:64] FLAG: --help="false" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231754 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231758 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231761 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231764 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231768 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231771 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231774 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231778 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231781 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231784 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231787 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231796 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231799 2571 flags.go:64] FLAG: --kube-reserved="" Jul 10 13:10:21.237601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231802 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231805 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231808 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231811 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231814 2571 flags.go:64] FLAG: --lock-file="" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231817 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231820 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231823 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231829 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231832 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231836 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231839 2571 flags.go:64] FLAG: --logging-format="text" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231843 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231847 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231850 2571 flags.go:64] FLAG: --manifest-url="" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231853 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231858 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231861 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231866 2571 flags.go:64] FLAG: --max-pods="110" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231874 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231877 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231880 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231883 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231886 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231890 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 13:10:21.238219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231893 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231901 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231904 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231907 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231910 2571 flags.go:64] FLAG: --pod-cidr="" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231913 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231929 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231932 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231936 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231939 2571 flags.go:64] FLAG: --port="10250" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231942 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231945 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e824f7f9fec23afe" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231948 2571 flags.go:64] FLAG: --qos-reserved="" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231951 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231954 2571 flags.go:64] FLAG: --register-node="true" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231957 2571 flags.go:64] FLAG: --register-with-taints="" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231967 2571 flags.go:64] FLAG: --registry-burst="10" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231970 2571 flags.go:64] FLAG: --registry-qps="5" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231973 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231976 2571 flags.go:64] FLAG: --reserved-memory="" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231981 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231984 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231987 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231990 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231994 2571 flags.go:64] FLAG: --runonce="false" Jul 10 13:10:21.238875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.231997 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232001 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232004 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232008 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232011 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232014 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232018 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232021 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232024 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232027 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232030 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232032 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232036 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232039 2571 flags.go:64] FLAG: --system-cgroups="" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232048 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232053 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232056 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232059 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232066 2571 flags.go:64] FLAG: --tls-min-version="" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232069 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232072 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232075 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232078 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232081 2571 flags.go:64] FLAG: --v="2" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232085 2571 flags.go:64] FLAG: --version="false" Jul 10 13:10:21.239588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232089 2571 flags.go:64] FLAG: --vmodule="" Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232093 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.232099 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232294 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232299 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232303 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232306 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232309 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232314 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232316 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232319 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232322 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232325 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232328 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232331 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232333 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232336 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232339 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232341 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232344 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 13:10:21.240592 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232347 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232350 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232358 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232361 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232364 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232367 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232369 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232372 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232375 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232377 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232380 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232383 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232385 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232388 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232392 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232395 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232398 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232400 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232403 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232406 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 13:10:21.241416 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232410 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232412 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232415 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232418 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232421 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232423 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232426 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232429 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232431 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232434 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232437 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232439 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232442 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232445 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232447 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232456 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232459 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232462 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232465 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232468 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 13:10:21.241933 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232470 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232473 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232477 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232482 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232486 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232489 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232493 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232497 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232500 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232503 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232506 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232509 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232513 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232516 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232519 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232521 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232524 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232527 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232530 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 13:10:21.242457 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232533 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232535 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232538 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232541 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232543 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232546 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232548 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232551 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232555 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232565 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232568 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232571 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232573 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232576 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232579 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232581 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232584 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232587 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232589 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 13:10:21.242947 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232593 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232602 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232605 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.232608 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.233461 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.242318 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.242335 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.242354 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.242365 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242427 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242434 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242437 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242441 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242445 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242448 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 13:10:21.243467 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242451 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242454 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242458 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242461 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242464 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242467 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242471 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242474 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242477 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242480 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242483 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242487 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242491 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242494 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242497 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242500 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242504 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242507 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242510 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 13:10:21.243846 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242514 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242518 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242521 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242524 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242528 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242530 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242533 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242536 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242539 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242542 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242544 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242547 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242550 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242553 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242555 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242558 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242561 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242564 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242567 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242569 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 13:10:21.244356 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242573 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242576 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242579 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242581 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242584 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242586 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242589 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242592 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242595 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242598 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242601 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242604 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242607 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242609 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242612 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242615 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242618 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242621 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242625 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 13:10:21.244879 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242629 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242632 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242635 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242638 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242640 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242643 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242645 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242648 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242651 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242654 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242656 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242659 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242661 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242664 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242667 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242670 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242673 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242676 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242678 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242681 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 13:10:21.245366 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242684 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242687 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242689 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242707 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242712 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242715 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242719 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242722 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242725 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242729 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242732 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242735 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242738 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242740 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242743 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 13:10:21.245875 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.242748 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242871 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242876 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242879 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242882 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242885 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242889 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242892 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242894 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242897 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242900 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242903 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242906 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242909 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242912 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242915 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242919 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242923 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242927 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242929 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 13:10:21.246282 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242932 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242937 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242940 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242943 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242946 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242949 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242952 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242955 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242958 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242961 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242965 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242969 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242972 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242975 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242978 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242980 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242983 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242985 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242988 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 13:10:21.246779 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242991 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242994 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242996 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.242999 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243002 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243004 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243007 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243011 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243013 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243016 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243019 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243022 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243024 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243027 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243030 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243033 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243035 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243038 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243041 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 13:10:21.247373 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243043 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243046 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243049 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243051 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243054 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243057 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243060 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243063 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243065 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243068 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243070 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243073 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243076 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243078 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243081 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243084 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243086 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243089 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243092 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243095 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 13:10:21.247851 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243097 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243101 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243104 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243106 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243109 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243112 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243114 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243117 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243119 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243122 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243125 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243143 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243148 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243151 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243154 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243157 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243160 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243162 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243165 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243168 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 13:10:21.248370 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243170 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:21.243173 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.243179 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.243283 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.245472 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.247111 2571 server.go:1019] "Starting client certificate rotation" Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.247223 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 13:10:21.248886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.247264 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 13:10:21.250120 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.250106 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 13:10:21.255138 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.255112 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 13:10:21.264174 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.264158 2571 log.go:25] "Validated CRI v1 runtime API" Jul 10 13:10:21.266449 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.266430 2571 log.go:25] "Validated CRI v1 image API" Jul 10 13:10:21.267430 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.267411 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 13:10:21.273235 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.273218 2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 86a7f2c5-51df-435e-9b3c-227ae5fc3fc1:/dev/nvme0n1p3 c0ebdeec-1914-4fcc-939e-cc61b460c515:/dev/nvme0n1p4] Jul 10 13:10:21.273296 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.273236 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 13:10:21.280280 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.280153 2571 manager.go:216] Machine: {Timestamp:2026-07-10 13:10:21.278793224 +0000 UTC m=+1.000719100 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100874 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26659c97aa3e3a74b0283926e6be37 SystemUUID:ec26659c-97aa-3e3a-74b0-283926e6be37 BootID:3c9eb7c7-d9ca-44a6-85c3-1abaf7085b86 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:f0:fb:b8:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:f0:fb:b8:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:cf:16:c5:91:91 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 13:10:21.280280 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.280267 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 13:10:21.280432 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.280346 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 13:10:21.280769 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.280749 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 13:10:21.283757 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.283731 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 13:10:21.283905 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.283760 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-196.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 13:10:21.285558 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.285544 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 13:10:21.285624 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.285562 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 13:10:21.285624 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.285571 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 13:10:21.286236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.286224 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 13:10:21.287747 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.287735 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 13:10:21.291675 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291662 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 13:10:21.291793 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291782 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 13:10:21.291847 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291813 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 10 13:10:21.291847 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291825 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 13:10:21.291935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291849 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 13:10:21.291935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291863 2571 kubelet.go:398] "Adding apiserver pod source" Jul 10 13:10:21.291935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.291874 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 13:10:21.292321 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.292308 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 13:10:21.292371 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.292328 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 13:10:21.297998 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.297978 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 13:10:21.300211 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300194 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 13:10:21.300291 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300235 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 13:10:21.300444 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300431 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 13:10:21.300514 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300450 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 13:10:21.300514 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300460 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 13:10:21.300514 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300470 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 13:10:21.300514 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300483 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 13:10:21.300514 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300492 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 13:10:21.300992 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300981 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 13:10:21.301049 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.300994 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 13:10:21.301049 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.301006 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 13:10:21.301049 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.301015 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 13:10:21.301049 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.301029 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 13:10:21.301049 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.301042 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 13:10:21.301290 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.301072 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 13:10:21.301290 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.301082 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 13:10:21.304406 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.304391 2571 server.go:1268] "Started kubelet" Jul 10 13:10:21.304501 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.304455 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 13:10:21.304623 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.304513 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 13:10:21.304623 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.304593 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 13:10:21.305321 ip-10-0-133-196 systemd[1]: Started Kubernetes Kubelet. Jul 10 13:10:21.306029 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.305901 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 13:10:21.306090 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.306067 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 10 13:10:21.307225 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.307208 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hbpqm" Jul 10 13:10:21.310324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.309965 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-196.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 13:10:21.310324 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.310072 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 13:10:21.310526 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.310481 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 13:10:21.311952 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.311932 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 13:10:21.314619 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.313382 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-196.ec2.internal.18c0efb6e89c5a56 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-196.ec2.internal,UID:ip-10-0-133-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-196.ec2.internal,},FirstTimestamp:2026-07-10 13:10:21.304363606 +0000 UTC m=+1.026289482,LastTimestamp:2026-07-10 13:10:21.304363606 +0000 UTC m=+1.026289482,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-196.ec2.internal,}" Jul 10 13:10:21.314756 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.314736 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 13:10:21.314880 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.314752 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 13:10:21.315648 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.315367 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.315748 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.315733 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 13:10:21.315882 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.315865 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 13:10:21.316014 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316001 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 13:10:21.316151 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.315981 2571 factory.go:55] Registering systemd factory Jul 10 13:10:21.316208 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316165 2571 factory.go:223] Registration of the systemd container factory successfully Jul 10 13:10:21.316253 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316214 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 10 13:10:21.316253 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316222 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 10 13:10:21.316406 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316391 2571 factory.go:153] Registering CRI-O factory Jul 10 13:10:21.316406 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316406 2571 factory.go:223] Registration of the crio container factory successfully Jul 10 13:10:21.316500 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316445 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 13:10:21.316500 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316463 2571 factory.go:103] Registering Raw factory Jul 10 13:10:21.316500 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316473 2571 manager.go:1196] Started watching for new ooms in manager Jul 10 13:10:21.316715 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316694 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hbpqm" Jul 10 13:10:21.316896 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.316884 2571 manager.go:349] Starting recovery of all containers Jul 10 13:10:21.320033 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.320006 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 13:10:21.320163 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.320070 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 13:10:21.329183 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.329167 2571 manager.go:354] Recovery completed Jul 10 13:10:21.335034 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.335019 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.336970 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.336953 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.337031 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.336980 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.337031 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.336991 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.337809 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.337792 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 13:10:21.337949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.337936 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 13:10:21.338077 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.338047 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 13:10:21.338397 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.337872 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.339607 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.339581 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.339685 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.339623 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.339685 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.339641 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.339767 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.339756 2571 policy_none.go:49] "None policy: Start" Jul 10 13:10:21.339818 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.339772 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 13:10:21.339818 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.339793 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 13:10:21.341343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.341327 2571 policy_none.go:47] "Start" Jul 10 13:10:21.372815 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.372680 2571 manager.go:337] "Starting Device Plugin manager" Jul 10 13:10:21.372904 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.372837 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 13:10:21.372904 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.372851 2571 server.go:87] "Starting device plugin registration server" Jul 10 13:10:21.373094 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.373079 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 13:10:21.373146 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.373116 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 13:10:21.373236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.373219 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 13:10:21.373331 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.373320 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 13:10:21.373331 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.373330 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 13:10:21.375285 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.375253 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 13:10:21.375361 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.375301 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.473897 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.473853 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.475365 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.474714 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.475365 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.474750 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.475365 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.474763 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.475365 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.474789 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.484274 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.484256 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.484360 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.484277 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-196.ec2.internal\": node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.496357 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.496339 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.500451 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.500427 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 13:10:21.501715 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.501691 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 13:10:21.501788 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.501734 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 13:10:21.501788 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.501760 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 13:10:21.501865 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.501800 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 13:10:21.504819 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.504802 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 13:10:21.597640 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.597605 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.602048 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.602020 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"] Jul 10 13:10:21.602150 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.602120 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.602970 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.602955 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.603036 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.602984 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.603036 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.602994 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.604857 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.604843 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.604992 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.604979 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.605030 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605013 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.605590 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605564 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.605678 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605599 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.605678 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605614 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.605678 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605625 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.605678 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605650 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.605678 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.605666 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.607027 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.607011 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.607087 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.607037 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 13:10:21.607724 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.607710 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory" Jul 10 13:10:21.607785 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.607736 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 13:10:21.607785 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.607751 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID" Jul 10 13:10:21.617348 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.617326 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cf7db545b083efad4fe99dd4fc40cef-config\") pod \"kube-apiserver-proxy-ip-10-0-133-196.ec2.internal\" (UID: \"2cf7db545b083efad4fe99dd4fc40cef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.617432 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.617352 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b9d94f267d0fea43cfeae11eb4744a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"8b9d94f267d0fea43cfeae11eb4744a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.617432 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.617373 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b9d94f267d0fea43cfeae11eb4744a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"8b9d94f267d0fea43cfeae11eb4744a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.629758 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.629740 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-196.ec2.internal\" not found" node="ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.634287 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.634270 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-196.ec2.internal\" not found" node="ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.697851 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.697825 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.718304 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.718279 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b9d94f267d0fea43cfeae11eb4744a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"8b9d94f267d0fea43cfeae11eb4744a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.718417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.718311 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b9d94f267d0fea43cfeae11eb4744a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"8b9d94f267d0fea43cfeae11eb4744a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.718417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.718350 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cf7db545b083efad4fe99dd4fc40cef-config\") pod \"kube-apiserver-proxy-ip-10-0-133-196.ec2.internal\" (UID: \"2cf7db545b083efad4fe99dd4fc40cef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.718417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.718378 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b9d94f267d0fea43cfeae11eb4744a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"8b9d94f267d0fea43cfeae11eb4744a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.718417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.718410 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b9d94f267d0fea43cfeae11eb4744a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"8b9d94f267d0fea43cfeae11eb4744a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.718624 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.718459 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cf7db545b083efad4fe99dd4fc40cef-config\") pod \"kube-apiserver-proxy-ip-10-0-133-196.ec2.internal\" (UID: \"2cf7db545b083efad4fe99dd4fc40cef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.798872 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.798813 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.905308 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:21.905274 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:21.931603 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.931579 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:21.938144 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:21.938122 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" Jul 10 13:10:22.007148 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.007123 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.107750 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.107671 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.208739 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.208694 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.247847 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.247808 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 13:10:22.248347 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.247988 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 13:10:22.309425 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.309391 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.320181 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.320132 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 13:05:21 +0000 UTC" deadline="2028-01-04 16:56:09.589034483 +0000 UTC" Jul 10 13:10:22.320181 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.320176 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13035h45m47.268861823s" Jul 10 13:10:22.400796 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.400759 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 13:10:22.410194 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.410154 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.484673 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:22.484638 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf7db545b083efad4fe99dd4fc40cef.slice/crio-4d338cf2301a5a2a41dbbb2c13907e202cbf944e13fb25d1dbc50f847d934a75 WatchSource:0}: Error finding container 4d338cf2301a5a2a41dbbb2c13907e202cbf944e13fb25d1dbc50f847d934a75: Status 404 returned error can't find the container with id 4d338cf2301a5a2a41dbbb2c13907e202cbf944e13fb25d1dbc50f847d934a75 Jul 10 13:10:22.485108 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:22.485079 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9d94f267d0fea43cfeae11eb4744a0.slice/crio-5ebfff91b05993eaf9223ef04537e33129fd15b954c4c713c5cc0ee4dfd10787 WatchSource:0}: Error finding container 5ebfff91b05993eaf9223ef04537e33129fd15b954c4c713c5cc0ee4dfd10787: Status 404 returned error can't find the container with id 5ebfff91b05993eaf9223ef04537e33129fd15b954c4c713c5cc0ee4dfd10787 Jul 10 13:10:22.489680 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.489665 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 13:10:22.505976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.505483 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" event={"ID":"8b9d94f267d0fea43cfeae11eb4744a0","Type":"ContainerStarted","Data":"5ebfff91b05993eaf9223ef04537e33129fd15b954c4c713c5cc0ee4dfd10787"} Jul 10 13:10:22.507621 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.507329 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" event={"ID":"2cf7db545b083efad4fe99dd4fc40cef","Type":"ContainerStarted","Data":"4d338cf2301a5a2a41dbbb2c13907e202cbf944e13fb25d1dbc50f847d934a75"} Jul 10 13:10:22.510650 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.510631 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.616117 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:22.616087 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found" Jul 10 13:10:22.630458 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.630409 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 13:10:22.715714 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.715682 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" Jul 10 13:10:22.732243 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.732221 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 13:10:22.733237 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.733225 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" Jul 10 13:10:22.742378 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.742360 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 13:10:22.856814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:22.856788 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 13:10:23.190257 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.190054 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 13:10:23.293213 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.293185 2571 apiserver.go:52] "Watching apiserver" Jul 10 13:10:23.301830 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.301807 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 13:10:23.303889 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.303861 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25","openshift-dns/node-resolver-rnmbh","openshift-image-registry/node-ca-mv57f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal","openshift-multus/multus-additional-cni-plugins-b8gsw","openshift-multus/network-metrics-daemon-cd8lg","openshift-network-diagnostics/network-check-target-mlmb7","openshift-network-operator/iptables-alerter-7sz2s","kube-system/global-pull-secret-syncer-xl7w9","kube-system/konnectivity-agent-q2cr9","kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fjpms","openshift-multus/multus-5wbm9","openshift-ovn-kubernetes/ovnkube-node-b44jh"] Jul 10 13:10:23.305670 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.305650 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.307250 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.307223 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.308300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.308275 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 13:10:23.308431 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.308372 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9lm7b\"" Jul 10 13:10:23.309155 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.308556 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 13:10:23.309400 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.309360 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.310004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.309984 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.310122 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.310047 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lwwxw\"" Jul 10 13:10:23.310259 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.310243 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 13:10:23.310605 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.310588 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 13:10:23.310696 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.310615 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.310772 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.310746 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.311621 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.311601 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.311813 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.311796 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.311905 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.311888 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m8lbz\"" Jul 10 13:10:23.312093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.312043 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 13:10:23.313230 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.313055 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.314085 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.313878 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gd88g\"" Jul 10 13:10:23.314085 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.313905 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 13:10:23.314261 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.314156 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 13:10:23.314261 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.314191 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.314388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.314260 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.314388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.314288 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 13:10:23.314388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.314341 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 13:10:23.315444 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.315425 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 13:10:23.315534 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.315479 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 13:10:23.315534 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.315491 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 13:10:23.315656 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.315493 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6ww8h\"" Jul 10 13:10:23.318417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.318361 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.318512 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.318448 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:23.320281 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.320201 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.320281 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.320254 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:23.322021 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.322005 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:23.322112 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.322085 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:23.324648 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.324327 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.326971 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.326947 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.327186 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.327165 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.327271 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.327237 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v494x\"" Jul 10 13:10:23.334793 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.334764 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-run-netns\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.334881 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.334803 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-ovnkube-config\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335140 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335159 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335192 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/706332e4-b03e-48e4-b23e-32aa43dac6f1-cni-binary-copy\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335229 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-cni-multus\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335266 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-etc-kubernetes\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335292 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5904000-3a60-4cea-80c3-85a812a39930-serviceca\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335334 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrj5\" (UniqueName: \"kubernetes.io/projected/c5904000-3a60-4cea-80c3-85a812a39930-kube-api-access-vwrj5\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335366 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-etc-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335401 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335436 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-system-cni-dir\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/881293b5-931f-4d9e-a312-29e17abf5c52-agent-certs\") pod \"konnectivity-agent-q2cr9\" (UID: \"881293b5-931f-4d9e-a312-29e17abf5c52\") " pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-cnibin\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335545 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-os-release\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5904000-3a60-4cea-80c3-85a812a39930-host\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335736 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-cni-netd\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.335865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335861 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-env-overrides\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335894 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.335915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-os-release\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336078 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-conf-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336113 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-kubelet\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336145 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336176 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwjdp\" (UniqueName: \"kubernetes.io/projected/39465ff9-05c2-413e-9a82-344b2c9d5549-kube-api-access-qwjdp\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336255 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336326 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghs9h\" (UniqueName: \"kubernetes.io/projected/04aaf506-cb9a-43ce-9556-3c046193e00b-kube-api-access-ghs9h\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336417 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-k8s-cni-cncf-io\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336454 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-netns\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336486 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-daemon-config\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336527 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-systemd\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336563 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336599 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-kubelet\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336699 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-multus-certs\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336749 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-systemd-units\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.336814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336807 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-log-socket\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336836 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20158f5f-220b-47fe-9bd2-7099859065a6-ovn-node-metrics-cert\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336876 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgn88\" (UniqueName: \"kubernetes.io/projected/20158f5f-220b-47fe-9bd2-7099859065a6-kube-api-access-xgn88\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336919 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-cnibin\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.336969 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/881293b5-931f-4d9e-a312-29e17abf5c52-konnectivity-ca\") pod \"konnectivity-agent-q2cr9\" (UID: \"881293b5-931f-4d9e-a312-29e17abf5c52\") " pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsgr5\" (UniqueName: \"kubernetes.io/projected/706332e4-b03e-48e4-b23e-32aa43dac6f1-kube-api-access-vsgr5\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337025 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-var-lib-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337053 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-cni-bin\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337173 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337238 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-cni-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337305 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-socket-dir-parent\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337374 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-cni-bin\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337434 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-ovn\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-node-log\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337515 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-ovnkube-script-lib\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337551 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.337686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337587 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-system-cni-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.338324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337644 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-hostroot\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.338324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.337680 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-slash\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.339966 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.338853 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 13:10:23.339966 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.339013 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.339966 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.339155 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pnd2n\"" Jul 10 13:10:23.339966 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.339216 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.340997 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.340976 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.342813 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.342794 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.343256 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.343238 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.343837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.343670 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 13:10:23.343837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.343673 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.343958 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.343849 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z79gv\"" Jul 10 13:10:23.345054 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.345034 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-chfvv\"" Jul 10 13:10:23.345166 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.345040 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 13:10:23.345360 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.345340 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 13:10:23.355482 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.354608 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r4wmv" Jul 10 13:10:23.359635 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.359615 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r4wmv" Jul 10 13:10:23.416843 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.416820 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 13:10:23.437865 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.437839 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-sys\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.437989 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.437879 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/881293b5-931f-4d9e-a312-29e17abf5c52-agent-certs\") pod \"konnectivity-agent-q2cr9\" (UID: \"881293b5-931f-4d9e-a312-29e17abf5c52\") " pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.437989 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.437908 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.437989 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.437936 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-cni-netd\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.437987 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-env-overrides\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.437999 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438045 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8530fac2-8e52-489b-99bb-ea51d491256c-tmp-dir\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438082 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-cni-netd\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438070 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-systemd\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438135 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-tuned\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438209 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-conf-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438255 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-kubelet\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438262 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438331 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438357 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghs9h\" (UniqueName: \"kubernetes.io/projected/04aaf506-cb9a-43ce-9556-3c046193e00b-kube-api-access-ghs9h\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438392 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysctl-conf\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.438435 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:23.438450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438287 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-conf-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438519 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w6zt\" (UniqueName: \"kubernetes.io/projected/75b9b5f3-2d19-4008-ad5d-fa0950399cee-kube-api-access-6w6zt\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-k8s-cni-cncf-io\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438612 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-netns\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438692 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-env-overrides\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438698 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-daemon-config\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438726 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-lib-modules\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-var-lib-kubelet\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438762 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-netns\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.438806 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:10:23.938787111 +0000 UTC m=+3.660712994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-multus-certs\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-log-socket\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.438921 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgn88\" (UniqueName: \"kubernetes.io/projected/20158f5f-220b-47fe-9bd2-7099859065a6-kube-api-access-xgn88\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-log-socket\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439155 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-kubelet\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439182 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-cnibin\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.439218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439203 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-k8s-cni-cncf-io\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439226 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-socket-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439248 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-cnibin\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439280 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b05fa0c-f2b6-440b-950f-1d316453a43c-dbus\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439394 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/881293b5-931f-4d9e-a312-29e17abf5c52-konnectivity-ca\") pod \"konnectivity-agent-q2cr9\" (UID: \"881293b5-931f-4d9e-a312-29e17abf5c52\") " pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439445 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-cni-bin\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439503 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-daemon-config\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439512 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-cni-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439583 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-cni-bin\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439093 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-run-multus-certs\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439610 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-socket-dir-parent\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-cni-bin\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439720 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-socket-dir-parent\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439766 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-ovn\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439782 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-multus-cni-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.440086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439799 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8530fac2-8e52-489b-99bb-ea51d491256c-hosts-file\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439866 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-ovn\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439903 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-cni-bin\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-registration-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.439980 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/881293b5-931f-4d9e-a312-29e17abf5c52-konnectivity-ca\") pod \"konnectivity-agent-q2cr9\" (UID: \"881293b5-931f-4d9e-a312-29e17abf5c52\") " pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440007 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-sys-fs\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440028 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysctl-d\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440061 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-system-cni-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440085 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-run-netns\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440102 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-system-cni-dir\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440163 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-run-netns\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440188 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440193 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-device-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/706332e4-b03e-48e4-b23e-32aa43dac6f1-cni-binary-copy\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440351 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5904000-3a60-4cea-80c3-85a812a39930-serviceca\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440695 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-system-cni-dir\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440734 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440760 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440779 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6ct6\" (UniqueName: \"kubernetes.io/projected/ecc816bc-a5bb-48b0-aca2-12196885fefc-kube-api-access-l6ct6\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440791 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/706332e4-b03e-48e4-b23e-32aa43dac6f1-cni-binary-copy\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440807 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-cnibin\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440823 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-system-cni-dir\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-cnibin\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440925 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-os-release\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440958 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5904000-3a60-4cea-80c3-85a812a39930-host\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440980 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-os-release\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.440992 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47735744-bbe8-40ac-ae3e-fba9d97031a4-host-slash\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5904000-3a60-4cea-80c3-85a812a39930-host\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441104 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441139 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwjdp\" (UniqueName: \"kubernetes.io/projected/39465ff9-05c2-413e-9a82-344b2c9d5549-kube-api-access-qwjdp\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441245 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47735744-bbe8-40ac-ae3e-fba9d97031a4-iptables-alerter-script\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-host\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:23.441986 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441462 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-systemd\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441490 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4hb\" (UniqueName: \"kubernetes.io/projected/8530fac2-8e52-489b-99bb-ea51d491256c-kube-api-access-7v4hb\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-kubelet\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441540 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-systemd-units\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441583 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20158f5f-220b-47fe-9bd2-7099859065a6-ovn-node-metrics-cert\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441590 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-systemd\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysconfig\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441642 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-systemd-units\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441648 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b05fa0c-f2b6-440b-950f-1d316453a43c-kubelet-config\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441657 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39465ff9-05c2-413e-9a82-344b2c9d5549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441677 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsgr5\" (UniqueName: \"kubernetes.io/projected/706332e4-b03e-48e4-b23e-32aa43dac6f1-kube-api-access-vsgr5\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441685 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-kubelet\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441714 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-var-lib-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441742 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441767 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-etc-selinux\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441810 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7gq8\" (UniqueName: \"kubernetes.io/projected/47735744-bbe8-40ac-ae3e-fba9d97031a4-kube-api-access-g7gq8\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441835 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-modprobe-d\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.442916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441869 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-node-log\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441902 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-ovnkube-script-lib\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.441926 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-kubernetes\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442002 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-run\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442025 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b9b5f3-2d19-4008-ad5d-fa0950399cee-tmp\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-hostroot\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442072 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-var-lib-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442106 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-slash\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442143 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-ovnkube-config\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442229 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442253 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-host-slash\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442263 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-node-log\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442297 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-cni-multus\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442318 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-etc-kubernetes\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrj5\" (UniqueName: \"kubernetes.io/projected/c5904000-3a60-4cea-80c3-85a812a39930-kube-api-access-vwrj5\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-etc-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.443910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442401 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-os-release\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442456 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442512 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-host-var-lib-cni-multus\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442529 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5904000-3a60-4cea-80c3-85a812a39930-serviceca\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442541 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-etc-kubernetes\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442547 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39465ff9-05c2-413e-9a82-344b2c9d5549-os-release\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442548 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/706332e4-b03e-48e4-b23e-32aa43dac6f1-hostroot\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442573 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-run-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442575 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20158f5f-220b-47fe-9bd2-7099859065a6-etc-openvswitch\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.442751 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-ovnkube-config\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.443112 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20158f5f-220b-47fe-9bd2-7099859065a6-ovnkube-script-lib\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.445051 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.444886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/881293b5-931f-4d9e-a312-29e17abf5c52-agent-certs\") pod \"konnectivity-agent-q2cr9\" (UID: \"881293b5-931f-4d9e-a312-29e17abf5c52\") " pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.447270 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.447207 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20158f5f-220b-47fe-9bd2-7099859065a6-ovn-node-metrics-cert\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.448720 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.448619 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghs9h\" (UniqueName: \"kubernetes.io/projected/04aaf506-cb9a-43ce-9556-3c046193e00b-kube-api-access-ghs9h\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.450590 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.450545 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgn88\" (UniqueName: \"kubernetes.io/projected/20158f5f-220b-47fe-9bd2-7099859065a6-kube-api-access-xgn88\") pod \"ovnkube-node-b44jh\" (UID: \"20158f5f-220b-47fe-9bd2-7099859065a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.450692 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.450594 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwjdp\" (UniqueName: \"kubernetes.io/projected/39465ff9-05c2-413e-9a82-344b2c9d5549-kube-api-access-qwjdp\") pod \"multus-additional-cni-plugins-b8gsw\" (UID: \"39465ff9-05c2-413e-9a82-344b2c9d5549\") " pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.451089 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.451066 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrj5\" (UniqueName: \"kubernetes.io/projected/c5904000-3a60-4cea-80c3-85a812a39930-kube-api-access-vwrj5\") pod \"node-ca-mv57f\" (UID: \"c5904000-3a60-4cea-80c3-85a812a39930\") " pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.452688 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.452646 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsgr5\" (UniqueName: \"kubernetes.io/projected/706332e4-b03e-48e4-b23e-32aa43dac6f1-kube-api-access-vsgr5\") pod \"multus-5wbm9\" (UID: \"706332e4-b03e-48e4-b23e-32aa43dac6f1\") " pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.542935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.542905 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4hb\" (UniqueName: \"kubernetes.io/projected/8530fac2-8e52-489b-99bb-ea51d491256c-kube-api-access-7v4hb\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.543119 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.542944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysconfig\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543119 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.542970 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b05fa0c-f2b6-440b-950f-1d316453a43c-kubelet-config\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.543119 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.542997 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-etc-selinux\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.543286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543132 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7gq8\" (UniqueName: \"kubernetes.io/projected/47735744-bbe8-40ac-ae3e-fba9d97031a4-kube-api-access-g7gq8\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.543286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543174 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-modprobe-d\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-kubernetes\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543228 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-run\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543236 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysconfig\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543253 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b9b5f3-2d19-4008-ad5d-fa0950399cee-tmp\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543293 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-sys\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543311 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b05fa0c-f2b6-440b-950f-1d316453a43c-kubelet-config\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543323 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-etc-selinux\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543334 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-modprobe-d\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543327 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8530fac2-8e52-489b-99bb-ea51d491256c-tmp-dir\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543355 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-run\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-sys\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-systemd\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-tuned\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543441 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-kubernetes\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543477 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-systemd\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543512 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysctl-conf\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w6zt\" (UniqueName: \"kubernetes.io/projected/75b9b5f3-2d19-4008-ad5d-fa0950399cee-kube-api-access-6w6zt\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-lib-modules\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.543586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-var-lib-kubelet\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543618 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-socket-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543640 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b05fa0c-f2b6-440b-950f-1d316453a43c-dbus\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543647 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8530fac2-8e52-489b-99bb-ea51d491256c-tmp-dir\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543708 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b05fa0c-f2b6-440b-950f-1d316453a43c-dbus\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543717 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysctl-conf\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543757 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-lib-modules\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543778 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-var-lib-kubelet\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543822 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-socket-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543873 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543919 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8530fac2-8e52-489b-99bb-ea51d491256c-hosts-file\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543954 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-registration-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.543980 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-sys-fs\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.544024 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544064 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysctl-d\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.544105 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:10:24.044085364 +0000 UTC m=+3.766011242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544142 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-device-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.544258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544144 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-registration-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544193 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-sysctl-d\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6ct6\" (UniqueName: \"kubernetes.io/projected/ecc816bc-a5bb-48b0-aca2-12196885fefc-kube-api-access-l6ct6\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544234 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47735744-bbe8-40ac-ae3e-fba9d97031a4-host-slash\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544240 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-device-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47735744-bbe8-40ac-ae3e-fba9d97031a4-iptables-alerter-script\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544265 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-sys-fs\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544286 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-host\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544312 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544322 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47735744-bbe8-40ac-ae3e-fba9d97031a4-host-slash\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544203 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8530fac2-8e52-489b-99bb-ea51d491256c-hosts-file\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544415 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75b9b5f3-2d19-4008-ad5d-fa0950399cee-host\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544502 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecc816bc-a5bb-48b0-aca2-12196885fefc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.545093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.544924 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47735744-bbe8-40ac-ae3e-fba9d97031a4-iptables-alerter-script\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.545912 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.545878 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b9b5f3-2d19-4008-ad5d-fa0950399cee-tmp\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.546083 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.546051 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75b9b5f3-2d19-4008-ad5d-fa0950399cee-etc-tuned\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.551416 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.551396 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7gq8\" (UniqueName: \"kubernetes.io/projected/47735744-bbe8-40ac-ae3e-fba9d97031a4-kube-api-access-g7gq8\") pod \"iptables-alerter-7sz2s\" (UID: \"47735744-bbe8-40ac-ae3e-fba9d97031a4\") " pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.551505 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.551482 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4hb\" (UniqueName: \"kubernetes.io/projected/8530fac2-8e52-489b-99bb-ea51d491256c-kube-api-access-7v4hb\") pod \"node-resolver-rnmbh\" (UID: \"8530fac2-8e52-489b-99bb-ea51d491256c\") " pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.553714 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.553694 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:23.553812 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.553720 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:23.553812 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.553733 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:23.553812 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.553795 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:10:24.053777196 +0000 UTC m=+3.775703077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:23.555974 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.555956 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6ct6\" (UniqueName: \"kubernetes.io/projected/ecc816bc-a5bb-48b0-aca2-12196885fefc-kube-api-access-l6ct6\") pod \"aws-ebs-csi-driver-node-btq25\" (UID: \"ecc816bc-a5bb-48b0-aca2-12196885fefc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.556082 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.556059 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w6zt\" (UniqueName: \"kubernetes.io/projected/75b9b5f3-2d19-4008-ad5d-fa0950399cee-kube-api-access-6w6zt\") pod \"tuned-fjpms\" (UID: \"75b9b5f3-2d19-4008-ad5d-fa0950399cee\") " pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.620027 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.620000 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:23.636070 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.636037 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5wbm9" Jul 10 13:10:23.646612 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.646590 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mv57f" Jul 10 13:10:23.658104 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.658059 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:23.666403 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.666385 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" Jul 10 13:10:23.675902 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.675881 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rnmbh" Jul 10 13:10:23.683518 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.683502 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" Jul 10 13:10:23.692518 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.692471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7sz2s" Jul 10 13:10:23.698417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.698397 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fjpms" Jul 10 13:10:23.948037 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:23.947956 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:23.948209 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.948105 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:23.948209 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:23.948193 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:10:24.94814481 +0000 UTC m=+4.670070690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:24.048630 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.048597 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:24.048814 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.048763 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:24.048869 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.048837 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:10:25.048811523 +0000 UTC m=+4.770737393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:24.149701 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.149669 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:24.149878 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.149860 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:24.149946 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.149883 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:24.149946 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.149895 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:24.150043 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.149953 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:10:25.149935797 +0000 UTC m=+4.871861668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:24.361967 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.361891 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 13:05:23 +0000 UTC" deadline="2028-01-01 01:01:13.571352254 +0000 UTC" Jul 10 13:10:24.361967 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.361916 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12947h50m49.209439533s" Jul 10 13:10:24.405034 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:24.404993 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20158f5f_220b_47fe_9bd2_7099859065a6.slice/crio-35576b1592ef98edbee4f0be4996e8126a9182cbb1e9b11100b23eb9ec49d041 WatchSource:0}: Error finding container 35576b1592ef98edbee4f0be4996e8126a9182cbb1e9b11100b23eb9ec49d041: Status 404 returned error can't find the container with id 35576b1592ef98edbee4f0be4996e8126a9182cbb1e9b11100b23eb9ec49d041 Jul 10 13:10:24.407235 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:24.407157 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881293b5_931f_4d9e_a312_29e17abf5c52.slice/crio-08bb5799cb1e7c39867667957155286ccf1a281edb004549b7103ce6c4b1048e WatchSource:0}: Error finding container 08bb5799cb1e7c39867667957155286ccf1a281edb004549b7103ce6c4b1048e: Status 404 returned error can't find the container with id 08bb5799cb1e7c39867667957155286ccf1a281edb004549b7103ce6c4b1048e Jul 10 13:10:24.413460 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:10:24.413433 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39465ff9_05c2_413e_9a82_344b2c9d5549.slice/crio-d1e15e4b326db1825380f65a033bf58b29fe3d564bcd345daf38e5db2759be5f WatchSource:0}: Error finding container d1e15e4b326db1825380f65a033bf58b29fe3d564bcd345daf38e5db2759be5f: Status 404 returned error can't find the container with id d1e15e4b326db1825380f65a033bf58b29fe3d564bcd345daf38e5db2759be5f Jul 10 13:10:24.502197 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.502052 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:24.502300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.502061 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:24.502300 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.502284 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:24.502393 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.502374 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:24.514923 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.514900 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mv57f" event={"ID":"c5904000-3a60-4cea-80c3-85a812a39930","Type":"ContainerStarted","Data":"ba29c8f3c3ce5afb6e685745ae13f34f79923e499047fa04dacf81da8c7213a6"} Jul 10 13:10:24.520145 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.520109 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q2cr9" event={"ID":"881293b5-931f-4d9e-a312-29e17abf5c52","Type":"ContainerStarted","Data":"08bb5799cb1e7c39867667957155286ccf1a281edb004549b7103ce6c4b1048e"} Jul 10 13:10:24.524572 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.524550 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"35576b1592ef98edbee4f0be4996e8126a9182cbb1e9b11100b23eb9ec49d041"} Jul 10 13:10:24.528490 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.528468 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" event={"ID":"2cf7db545b083efad4fe99dd4fc40cef","Type":"ContainerStarted","Data":"1a5a8cbb5146ee0e254067f1572859380e1975f97e93be033c27175fcfa19e2a"} Jul 10 13:10:24.530335 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.530315 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rnmbh" event={"ID":"8530fac2-8e52-489b-99bb-ea51d491256c","Type":"ContainerStarted","Data":"ab1f2b6dcc49d043314f6667e31f80550117354c45d50191914dc1dd03ef043f"} Jul 10 13:10:24.531466 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.531443 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fjpms" event={"ID":"75b9b5f3-2d19-4008-ad5d-fa0950399cee","Type":"ContainerStarted","Data":"8aede75d71e7c5a48f3ac9d19133b9982cda4f3d08e88db2365e2b8d88241a30"} Jul 10 13:10:24.534372 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.534349 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5wbm9" event={"ID":"706332e4-b03e-48e4-b23e-32aa43dac6f1","Type":"ContainerStarted","Data":"3d914aeed1022cf2d47fedc4027b6df547e5cac2d79a9f26086bf00b1222492d"} Jul 10 13:10:24.535375 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.535355 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerStarted","Data":"d1e15e4b326db1825380f65a033bf58b29fe3d564bcd345daf38e5db2759be5f"} Jul 10 13:10:24.537671 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.537649 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" event={"ID":"ecc816bc-a5bb-48b0-aca2-12196885fefc","Type":"ContainerStarted","Data":"0398e2003e63d4ed8e2083230d02399b51677e97fb8625531cb1cc04fa4a0f66"} Jul 10 13:10:24.538912 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.538893 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7sz2s" event={"ID":"47735744-bbe8-40ac-ae3e-fba9d97031a4","Type":"ContainerStarted","Data":"aa2d2b739fa3454f7dded4251eda9a0827e7c4d6946db78bcd0996fc5eb56776"} Jul 10 13:10:24.544822 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.544761 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" podStartSLOduration=2.54474466 podStartE2EDuration="2.54474466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:24.544670147 +0000 UTC m=+4.266596032" watchObservedRunningTime="2026-07-10 13:10:24.54474466 +0000 UTC m=+4.266670550" Jul 10 13:10:24.954040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:24.953996 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:24.954274 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.954254 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:24.954360 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:24.954334 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:10:26.954312132 +0000 UTC m=+6.676238009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:25.055477 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.054802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:25.055477 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.055028 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:25.055477 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.055116 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:10:27.055076827 +0000 UTC m=+6.777002688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:25.156231 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.155467 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:25.156231 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.155714 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:25.156231 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.155735 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:25.156231 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.155748 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:25.156231 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.155808 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:10:27.15578893 +0000 UTC m=+6.877714811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:25.362629 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.362535 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 13:05:23 +0000 UTC" deadline="2028-01-22 10:45:33.975229867 +0000 UTC" Jul 10 13:10:25.362629 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.362599 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13461h35m8.612634896s" Jul 10 13:10:25.503154 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.503064 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:25.503358 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:25.503218 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:25.557794 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.557728 2571 generic.go:356] "Generic (PLEG): container finished" podID="8b9d94f267d0fea43cfeae11eb4744a0" containerID="43ab5bb7d2a4887ea5649aa0628f38d7dad9f24967e50fef725d0dc83191eb37" exitCode=0 Jul 10 13:10:25.558726 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:25.558696 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" event={"ID":"8b9d94f267d0fea43cfeae11eb4744a0","Type":"ContainerDied","Data":"43ab5bb7d2a4887ea5649aa0628f38d7dad9f24967e50fef725d0dc83191eb37"} Jul 10 13:10:26.502965 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:26.502931 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:26.503422 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:26.503048 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:26.503486 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:26.503469 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:26.503788 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:26.503599 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:26.566131 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:26.565480 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" event={"ID":"8b9d94f267d0fea43cfeae11eb4744a0","Type":"ContainerStarted","Data":"b95f9d3c8f9624e97e352f7383e35d72ed2412d2d228179754101ebeb5e49c56"} Jul 10 13:10:26.973721 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:26.973639 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:26.973881 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:26.973848 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:26.973954 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:26.973910 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:10:30.973890924 +0000 UTC m=+10.695816799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:27.074502 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:27.074274 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:27.075322 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.074837 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:27.075322 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.074919 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:10:31.074899562 +0000 UTC m=+10.796825429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:27.175694 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:27.175658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:27.175861 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.175829 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:27.175861 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.175851 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:27.175861 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.175862 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:27.176020 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.175920 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:10:31.17590255 +0000 UTC m=+10.897828410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:27.504505 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:27.504473 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:27.504941 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:27.504621 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:28.502506 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:28.502465 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:28.502663 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:28.502595 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:28.502936 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:28.502912 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:28.503073 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:28.503037 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:29.502862 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:29.502631 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:29.502862 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:29.502753 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:30.502833 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:30.502805 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:30.502998 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:30.502930 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:30.502998 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:30.502977 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:30.503416 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:30.503132 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:31.010113 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:31.009998 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:31.010288 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.010134 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:31.010288 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.010195 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:10:39.010176577 +0000 UTC m=+18.732102437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:31.111087 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:31.111036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:31.111270 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.111184 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:31.111413 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.111319 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:10:39.111299602 +0000 UTC m=+18.833225473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:31.212394 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:31.212274 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:31.212574 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.212458 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:31.212574 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.212486 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:31.212574 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.212500 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:31.212574 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.212560 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:10:39.212542772 +0000 UTC m=+18.934468649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:31.503584 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:31.503495 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:31.504101 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:31.503604 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:32.502005 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:32.501961 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:32.502190 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:32.502013 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:32.502190 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:32.502116 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:32.502270 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:32.502230 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:33.502517 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:33.501997 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:33.502517 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:33.502164 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:34.503118 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:34.503082 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:34.503614 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:34.503123 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:34.503614 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:34.503209 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:34.503614 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:34.503360 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:35.502791 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:35.502733 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:35.502897 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:35.502834 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:36.502434 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:36.502405 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:36.502893 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:36.502527 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:36.502893 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:36.502580 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:36.502893 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:36.502684 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:37.502065 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:37.502014 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:37.502259 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:37.502178 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:38.502946 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:38.502902 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:38.503522 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:38.502909 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:38.503522 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:38.503033 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:38.503522 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:38.503137 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:39.071036 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:39.071003 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:39.071210 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.071171 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:39.071258 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.071231 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:10:55.071215416 +0000 UTC m=+34.793141277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:39.172013 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:39.171981 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:39.172137 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.172124 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:39.172187 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.172178 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:10:55.17216258 +0000 UTC m=+34.894088444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:39.272605 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:39.272563 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:39.272743 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.272723 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:39.272783 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.272751 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:39.272783 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.272766 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:39.272868 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.272828 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:10:55.27281022 +0000 UTC m=+34.994736095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:39.502236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:39.502151 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:39.502389 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:39.502274 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:40.502159 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:40.502119 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:40.502614 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:40.502122 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:40.502614 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:40.502257 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:40.502614 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:40.502299 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:41.503884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:41.503848 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:41.504304 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:41.503961 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:42.502067 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:42.502014 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:42.502262 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:42.502159 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:42.502262 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:42.502216 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:42.502380 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:42.502338 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:43.502407 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:43.502381 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:43.502787 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:43.502484 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:44.502622 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.502285 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:44.503420 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.502344 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:44.503420 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:44.502666 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:44.503420 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:44.502745 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:44.605412 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.605381 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q2cr9" event={"ID":"881293b5-931f-4d9e-a312-29e17abf5c52","Type":"ContainerStarted","Data":"c6728de288a7f1af0ffbb81f2d91bee1cad03d385c2fc145345d0716ef2fa045"} Jul 10 13:10:44.607977 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.607958 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:10:44.608304 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608264 2571 generic.go:356] "Generic (PLEG): container finished" podID="20158f5f-220b-47fe-9bd2-7099859065a6" containerID="bddb097b7256726548702c19e961805ed26ee5751f4143c2d7f6de13f47e102d" exitCode=1 Jul 10 13:10:44.608423 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608327 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"869942377e6c928f74945ec25494dac9579b7be6884922686cd03f1971d2edf6"} Jul 10 13:10:44.608423 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608405 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"015f0a3da1bc331afb42409d163b3797371ddd3e8c74b91021e50ca934aaca95"} Jul 10 13:10:44.608550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608458 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"9bd1f4ee8d27ce92aeb1aa78b470a16837395e28b26e1efc2d77949e25e37f62"} Jul 10 13:10:44.608550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608498 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"167c12feab96c8a381c7772d658c6afd495b082095a03e47fabc8ffaa3d92756"} Jul 10 13:10:44.608550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608528 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerDied","Data":"bddb097b7256726548702c19e961805ed26ee5751f4143c2d7f6de13f47e102d"} Jul 10 13:10:44.608658 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.608558 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"5ba35eb3c78520233a1587ce922df3c38af43ec798297aa29e59be56e863d068"} Jul 10 13:10:44.609675 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.609655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rnmbh" event={"ID":"8530fac2-8e52-489b-99bb-ea51d491256c","Type":"ContainerStarted","Data":"8aa5558577c6fd43c3180905ad88731de09fda1f032af42c05d07536a67776f5"} Jul 10 13:10:44.610990 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.610971 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fjpms" event={"ID":"75b9b5f3-2d19-4008-ad5d-fa0950399cee","Type":"ContainerStarted","Data":"f1038a217a997258d4ae5d05607672743b4d3b130aeb5de8a31f6c15a1c9cf87"} Jul 10 13:10:44.612330 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.612295 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5wbm9" event={"ID":"706332e4-b03e-48e4-b23e-32aa43dac6f1","Type":"ContainerStarted","Data":"9ff8d651df46e63f212b5af818a80b487ae8c0304424070025a9439fa05bfb6d"} Jul 10 13:10:44.613615 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.613597 2571 generic.go:356] "Generic (PLEG): container finished" podID="39465ff9-05c2-413e-9a82-344b2c9d5549" containerID="947a4b9f2556e46d2149adbf41acae1ad8024735e2d56b2f38754996c4ae30e6" exitCode=0 Jul 10 13:10:44.613710 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.613670 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerDied","Data":"947a4b9f2556e46d2149adbf41acae1ad8024735e2d56b2f38754996c4ae30e6"} Jul 10 13:10:44.616727 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.616701 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" event={"ID":"ecc816bc-a5bb-48b0-aca2-12196885fefc","Type":"ContainerStarted","Data":"9c618edd35f0d1e4b440fcb25e593300b8e40b48416f5e3503239ed4a74c9d5d"} Jul 10 13:10:44.618043 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.618021 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mv57f" event={"ID":"c5904000-3a60-4cea-80c3-85a812a39930","Type":"ContainerStarted","Data":"40feb3eb20fb6da60a6772b8968a3c74650909c61d84a907b486985a644160fc"} Jul 10 13:10:44.620348 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.620314 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" podStartSLOduration=22.620299419 podStartE2EDuration="22.620299419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:26.580914833 +0000 UTC m=+6.302840717" watchObservedRunningTime="2026-07-10 13:10:44.620299419 +0000 UTC m=+24.342225302" Jul 10 13:10:44.620644 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.620616 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q2cr9" podStartSLOduration=8.870487204 podStartE2EDuration="23.620609794s" totalImagesPullingTime="14.75012259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:44.620437336 +0000 UTC m=+24.342363217" watchObservedRunningTime="2026-07-10 13:10:44.620609794 +0000 UTC m=+24.342535696" Jul 10 13:10:44.638131 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.638096 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5wbm9" podStartSLOduration=4.493154571 podStartE2EDuration="23.638084757s" totalImagesPullingTime="19.144930186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:44.637715323 +0000 UTC m=+24.359641207" watchObservedRunningTime="2026-07-10 13:10:44.638084757 +0000 UTC m=+24.360010638" Jul 10 13:10:44.673775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.673739 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mv57f" podStartSLOduration=4.590054499 podStartE2EDuration="23.673727502s" totalImagesPullingTime="19.083673003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:44.673304368 +0000 UTC m=+24.395230274" watchObservedRunningTime="2026-07-10 13:10:44.673727502 +0000 UTC m=+24.395653387" Jul 10 13:10:44.689586 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.689552 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rnmbh" podStartSLOduration=4.605490807 podStartE2EDuration="23.689541056s" totalImagesPullingTime="19.084050249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:44.689377423 +0000 UTC m=+24.411303296" watchObservedRunningTime="2026-07-10 13:10:44.689541056 +0000 UTC m=+24.411466937" Jul 10 13:10:44.708993 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:44.708964 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fjpms" podStartSLOduration=4.607234052 podStartE2EDuration="23.708953682s" totalImagesPullingTime="19.10171963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:44.708743756 +0000 UTC m=+24.430669637" watchObservedRunningTime="2026-07-10 13:10:44.708953682 +0000 UTC m=+24.430879563" Jul 10 13:10:45.191231 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.191204 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 13:10:45.241187 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.241166 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:45.241718 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.241702 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:45.384660 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.384524 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T13:10:45.191221649Z","UUID":"6a0e2e36-d939-4082-a3c8-6559b7eadfe1","Handler":null,"Name":"","Endpoint":""} Jul 10 13:10:45.385896 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.385873 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 13:10:45.386008 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.385914 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 13:10:45.502917 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.502890 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:45.503301 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:45.502989 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:45.621550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.621520 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" event={"ID":"ecc816bc-a5bb-48b0-aca2-12196885fefc","Type":"ContainerStarted","Data":"19d99c9972e62aeeda898d9d8ce4df8aee75ed9867e78e6c4ee6b9683cc0d214"} Jul 10 13:10:45.622999 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.622975 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7sz2s" event={"ID":"47735744-bbe8-40ac-ae3e-fba9d97031a4","Type":"ContainerStarted","Data":"216b3544816643a2a61952bbd7805dc53e1de1b8b169d0d1c609fcedc660ea18"} Jul 10 13:10:45.623528 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.623511 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:45.623945 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.623928 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q2cr9" Jul 10 13:10:45.637432 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:45.637361 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7sz2s" podStartSLOduration=5.556826872 podStartE2EDuration="24.637346119s" totalImagesPullingTime="19.080519247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:45.637340334 +0000 UTC m=+25.359266219" watchObservedRunningTime="2026-07-10 13:10:45.637346119 +0000 UTC m=+25.359272001" Jul 10 13:10:46.502625 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:46.502372 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:46.502754 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:46.502383 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:46.502754 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:46.502715 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:46.502853 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:46.502760 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:46.627964 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:46.627942 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:10:46.628358 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:46.628312 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"defe6c7febc417ef1184acdd364b2d2752d96409c4316de6fe3740422a1ed8f6"} Jul 10 13:10:47.502861 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:47.502832 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:47.503181 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:47.502969 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:47.634107 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:47.634048 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" event={"ID":"ecc816bc-a5bb-48b0-aca2-12196885fefc","Type":"ContainerStarted","Data":"e79afd5e3e3f9dac369e6f2f54cd44d0501b1b3a9aaf3f62868c19e05f22c728"} Jul 10 13:10:47.652005 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:47.651958 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btq25" podStartSLOduration=4.800664125 podStartE2EDuration="26.651943752s" totalImagesPullingTime="21.851279627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:47.651893464 +0000 UTC m=+27.373819347" watchObservedRunningTime="2026-07-10 13:10:47.651943752 +0000 UTC m=+27.373869612" Jul 10 13:10:48.502771 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:48.502732 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:48.502932 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:48.502740 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:48.502932 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:48.502855 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:48.503040 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:48.502947 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:49.502171 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:49.502139 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:49.502923 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:49.502260 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:50.502752 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.502587 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:50.503150 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.502643 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:50.503150 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:50.502867 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:50.503150 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:50.502910 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:50.640828 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.640772 2571 generic.go:356] "Generic (PLEG): container finished" podID="39465ff9-05c2-413e-9a82-344b2c9d5549" containerID="a507a025c94ccc240837a32b261414aaf6b0a9fd80f4a9ed069d57eb62db5232" exitCode=0 Jul 10 13:10:50.640926 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.640862 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerDied","Data":"a507a025c94ccc240837a32b261414aaf6b0a9fd80f4a9ed069d57eb62db5232"} Jul 10 13:10:50.644342 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.644326 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:10:50.644646 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.644628 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"d2072bc49bf6f5b50ba5a986713e710ab989efb663bf91c7a8e4e38fe26c13f5"} Jul 10 13:10:50.644925 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.644903 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:50.645007 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.644968 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:50.645007 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.645007 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:50.645130 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.645090 2571 scope.go:117] "RemoveContainer" containerID="bddb097b7256726548702c19e961805ed26ee5751f4143c2d7f6de13f47e102d" Jul 10 13:10:50.661549 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.661530 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:50.662295 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:50.662274 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:10:51.503093 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:51.503041 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:51.503656 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:51.503148 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:51.650335 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:51.650315 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:10:51.650689 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:51.650668 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" event={"ID":"20158f5f-220b-47fe-9bd2-7099859065a6","Type":"ContainerStarted","Data":"9636dfeeb126c693240683cddbcd5bb9bdd1b97c2706aa6a68a946a8232a543e"} Jul 10 13:10:51.677858 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:51.677813 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" podStartSLOduration=11.510991431 podStartE2EDuration="30.677800293s" totalImagesPullingTime="19.166808862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:10:51.676548153 +0000 UTC m=+31.398474034" watchObservedRunningTime="2026-07-10 13:10:51.677800293 +0000 UTC m=+31.399726196" Jul 10 13:10:52.061311 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.061280 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xl7w9"] Jul 10 13:10:52.061499 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.061424 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:52.061566 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:52.061518 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:52.062096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.062035 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mlmb7"] Jul 10 13:10:52.062232 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.062138 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:52.062293 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:52.062238 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:52.062584 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.062557 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cd8lg"] Jul 10 13:10:52.062695 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.062665 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:52.062795 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:52.062775 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:52.654457 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.654429 2571 generic.go:356] "Generic (PLEG): container finished" podID="39465ff9-05c2-413e-9a82-344b2c9d5549" containerID="332a9d6612a7b9e724f7bb3b113645041281c0f7649fb45ed7c099497e621741" exitCode=0 Jul 10 13:10:52.654791 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.654515 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerDied","Data":"332a9d6612a7b9e724f7bb3b113645041281c0f7649fb45ed7c099497e621741"} Jul 10 13:10:52.794719 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:52.794665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rnmbh_8530fac2-8e52-489b-99bb-ea51d491256c/dns-node-resolver/0.log" Jul 10 13:10:53.502509 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:53.502486 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:53.502612 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:53.502486 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:53.502612 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:53.502588 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:53.502707 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:53.502687 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:53.502741 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:53.502486 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:53.502805 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:53.502788 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:53.658523 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:53.658494 2571 generic.go:356] "Generic (PLEG): container finished" podID="39465ff9-05c2-413e-9a82-344b2c9d5549" containerID="222e10bbe282f644e4354cdb0c54b6ace2332b4b390c22dd663962101433d8cd" exitCode=0 Jul 10 13:10:53.659297 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:53.658541 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerDied","Data":"222e10bbe282f644e4354cdb0c54b6ace2332b4b390c22dd663962101433d8cd"} Jul 10 13:10:53.766776 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:53.766749 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mv57f_c5904000-3a60-4cea-80c3-85a812a39930/node-ca/0.log" Jul 10 13:10:55.080906 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:55.080872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:55.081355 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.081019 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:55.081355 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.081105 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs podName:04aaf506-cb9a-43ce-9556-3c046193e00b nodeName:}" failed. No retries permitted until 2026-07-10 13:11:27.081087319 +0000 UTC m=+66.803013179 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs") pod "network-metrics-daemon-cd8lg" (UID: "04aaf506-cb9a-43ce-9556-3c046193e00b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 13:10:55.181519 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:55.181446 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:55.181666 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.181575 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:55.181666 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.181631 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret podName:0b05fa0c-f2b6-440b-950f-1d316453a43c nodeName:}" failed. No retries permitted until 2026-07-10 13:11:27.181618005 +0000 UTC m=+66.903543868 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret") pod "global-pull-secret-syncer-xl7w9" (UID: "0b05fa0c-f2b6-440b-950f-1d316453a43c") : object "kube-system"/"original-pull-secret" not registered Jul 10 13:10:55.282133 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:55.282102 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:55.282283 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.282220 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 13:10:55.282283 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.282243 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 13:10:55.282283 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.282254 2571 projected.go:196] Error preparing data for projected volume kube-api-access-68jdn for pod openshift-network-diagnostics/network-check-target-mlmb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:55.282407 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.282300 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn podName:31f6fbb3-3804-415f-9755-0837ba776ada nodeName:}" failed. No retries permitted until 2026-07-10 13:11:27.282287144 +0000 UTC m=+67.004213003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-68jdn" (UniqueName: "kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn") pod "network-check-target-mlmb7" (UID: "31f6fbb3-3804-415f-9755-0837ba776ada") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 13:10:55.502101 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:55.502074 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:55.502262 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.502190 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:55.502262 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:55.502074 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:55.502389 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:55.502074 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:55.502389 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.502277 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:55.502475 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:55.502397 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:57.502954 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:57.502920 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:57.502954 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:57.502947 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:57.503495 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:57.502920 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:57.503495 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:57.503071 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:57.503495 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:57.503162 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:57.503495 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:57.503227 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:58.672719 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:58.672684 2571 generic.go:356] "Generic (PLEG): container finished" podID="39465ff9-05c2-413e-9a82-344b2c9d5549" containerID="7f46b5087a464d78059a0d4a4db237f7142a764ee55609a4503f077b32fa2ffa" exitCode=0 Jul 10 13:10:58.673183 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:58.672726 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerDied","Data":"7f46b5087a464d78059a0d4a4db237f7142a764ee55609a4503f077b32fa2ffa"} Jul 10 13:10:59.502400 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:59.502360 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:10:59.502549 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:59.502361 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:10:59.502549 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:59.502468 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:10:59.502656 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:59.502577 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:10:59.502656 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:59.502361 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:10:59.502771 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:10:59.502669 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:10:59.677824 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:59.677793 2571 generic.go:356] "Generic (PLEG): container finished" podID="39465ff9-05c2-413e-9a82-344b2c9d5549" containerID="f5598f293f2337c831976600db270e69a13b50e016b727027274e12b8c447af7" exitCode=0 Jul 10 13:10:59.678258 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:10:59.677838 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerDied","Data":"f5598f293f2337c831976600db270e69a13b50e016b727027274e12b8c447af7"} Jul 10 13:11:00.682760 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:00.682539 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" event={"ID":"39465ff9-05c2-413e-9a82-344b2c9d5549","Type":"ContainerStarted","Data":"43e89e166e3987d550d6d0cf1312dca8bfe16a524834124651f9226378ceb94f"} Jul 10 13:11:00.703476 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:00.703428 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b8gsw" podStartSLOduration=9.159772931 podStartE2EDuration="39.703414143s" totalImagesPullingTime="30.543641212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:00.703025603 +0000 UTC m=+40.424951487" watchObservedRunningTime="2026-07-10 13:11:00.703414143 +0000 UTC m=+40.425340025" Jul 10 13:11:01.503796 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:01.503767 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:01.503953 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:01.503870 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:01.503953 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:01.503883 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:01.503953 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:01.503915 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:01.504074 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:01.503962 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:01.504074 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:01.504029 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:03.502952 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:03.502923 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:03.503390 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:03.503012 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:03.503390 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:03.503135 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:03.503390 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:03.503016 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:03.503390 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:03.503170 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:03.503390 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:03.503232 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:05.502564 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:05.502535 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:05.502918 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:05.502535 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:05.502918 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:05.502639 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:05.502918 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:05.502749 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:05.502918 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:05.502540 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:05.502918 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:05.502856 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:07.502439 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:07.502407 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:07.503021 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:07.502412 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:07.503021 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:07.502508 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:07.503021 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:07.502417 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:07.503021 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:07.502574 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:07.503021 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:07.502645 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:09.501908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:09.501877 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:09.502283 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:09.501885 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:09.502283 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:09.502006 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:09.502283 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:09.501885 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:09.502283 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:09.502071 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:09.502283 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:09.502139 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:11.503381 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:11.503352 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:11.503814 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:11.503438 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:11.503814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:11.503538 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:11.503814 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:11.503646 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:11.503814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:11.503702 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:11.504011 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:11.503809 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:13.502413 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:13.502379 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:13.502844 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:13.502521 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:13.502844 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:13.502543 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:13.502844 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:13.502587 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:13.502844 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:13.502618 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:13.502844 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:13.502676 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:15.502343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:15.502307 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:15.502343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:15.502330 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:15.502833 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:15.502307 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:15.502833 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:15.502399 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:15.502833 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:15.502531 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:15.502833 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:15.502649 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:17.502755 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:17.502725 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:17.503224 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:17.502729 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:17.503224 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:17.502846 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cd8lg" podUID="04aaf506-cb9a-43ce-9556-3c046193e00b" Jul 10 13:11:17.503224 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:17.502738 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:17.503224 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:17.502931 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7w9" podUID="0b05fa0c-f2b6-440b-950f-1d316453a43c" Jul 10 13:11:17.503224 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:17.502999 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlmb7" podUID="31f6fbb3-3804-415f-9755-0837ba776ada" Jul 10 13:11:18.070313 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.070289 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeReady" Jul 10 13:11:18.070417 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.070411 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 13:11:18.133711 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.133683 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pfnk7"] Jul 10 13:11:18.137034 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.136409 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.139108 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.139084 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nmczh"] Jul 10 13:11:18.139446 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.139427 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrvn5\"" Jul 10 13:11:18.139842 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.139825 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 13:11:18.140209 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.140195 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 13:11:18.141184 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.141167 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.145045 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.145024 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 13:11:18.145153 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.145088 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 13:11:18.145216 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.145181 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ndhrp\"" Jul 10 13:11:18.145299 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.145282 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 13:11:18.156772 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.156753 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pfnk7"] Jul 10 13:11:18.157521 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.157502 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nmczh"] Jul 10 13:11:18.159806 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.159785 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dkrrp"] Jul 10 13:11:18.161832 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.161816 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.164175 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.164159 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4zgls\"" Jul 10 13:11:18.164255 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.164213 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 13:11:18.164505 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.164487 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 13:11:18.164910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.164893 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 13:11:18.166987 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.166885 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 13:11:18.183097 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.183073 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dkrrp"] Jul 10 13:11:18.248335 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248272 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5961e326-2362-41a5-84cb-0eb37fb30150-metrics-tls\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.248335 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248304 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7txq\" (UniqueName: \"kubernetes.io/projected/5961e326-2362-41a5-84cb-0eb37fb30150-kube-api-access-f7txq\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.248335 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248323 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7774828c-22a9-4793-8947-7a36a29db1a8-data-volume\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.248529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248366 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5961e326-2362-41a5-84cb-0eb37fb30150-config-volume\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.248529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248396 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7774828c-22a9-4793-8947-7a36a29db1a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.248529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f08d0d3e-3c8d-4eb3-8eef-786babdb96e7-cert\") pod \"ingress-canary-nmczh\" (UID: \"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7\") " pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.248529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5961e326-2362-41a5-84cb-0eb37fb30150-tmp-dir\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.248529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248493 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tf4r\" (UniqueName: \"kubernetes.io/projected/7774828c-22a9-4793-8947-7a36a29db1a8-kube-api-access-4tf4r\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.248529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248526 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js9ps\" (UniqueName: \"kubernetes.io/projected/f08d0d3e-3c8d-4eb3-8eef-786babdb96e7-kube-api-access-js9ps\") pod \"ingress-canary-nmczh\" (UID: \"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7\") " pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.248739 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7774828c-22a9-4793-8947-7a36a29db1a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.248739 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.248596 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7774828c-22a9-4793-8947-7a36a29db1a8-crio-socket\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.349819 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349789 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5961e326-2362-41a5-84cb-0eb37fb30150-metrics-tls\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.349819 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349819 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7txq\" (UniqueName: \"kubernetes.io/projected/5961e326-2362-41a5-84cb-0eb37fb30150-kube-api-access-f7txq\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7774828c-22a9-4793-8947-7a36a29db1a8-data-volume\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349858 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5961e326-2362-41a5-84cb-0eb37fb30150-config-volume\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7774828c-22a9-4793-8947-7a36a29db1a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f08d0d3e-3c8d-4eb3-8eef-786babdb96e7-cert\") pod \"ingress-canary-nmczh\" (UID: \"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7\") " pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5961e326-2362-41a5-84cb-0eb37fb30150-tmp-dir\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349934 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tf4r\" (UniqueName: \"kubernetes.io/projected/7774828c-22a9-4793-8947-7a36a29db1a8-kube-api-access-4tf4r\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349964 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js9ps\" (UniqueName: \"kubernetes.io/projected/f08d0d3e-3c8d-4eb3-8eef-786babdb96e7-kube-api-access-js9ps\") pod \"ingress-canary-nmczh\" (UID: \"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7\") " pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.350009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.349989 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7774828c-22a9-4793-8947-7a36a29db1a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350405 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.350024 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7774828c-22a9-4793-8947-7a36a29db1a8-crio-socket\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350405 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.350191 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7774828c-22a9-4793-8947-7a36a29db1a8-data-volume\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350405 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.350241 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7774828c-22a9-4793-8947-7a36a29db1a8-crio-socket\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350617 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.350594 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7774828c-22a9-4793-8947-7a36a29db1a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.350696 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.350596 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5961e326-2362-41a5-84cb-0eb37fb30150-config-volume\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.350739 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.350726 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5961e326-2362-41a5-84cb-0eb37fb30150-tmp-dir\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.353902 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.353878 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5961e326-2362-41a5-84cb-0eb37fb30150-metrics-tls\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.354007 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.353945 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7774828c-22a9-4793-8947-7a36a29db1a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.354103 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.354084 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f08d0d3e-3c8d-4eb3-8eef-786babdb96e7-cert\") pod \"ingress-canary-nmczh\" (UID: \"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7\") " pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.358307 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.358281 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tf4r\" (UniqueName: \"kubernetes.io/projected/7774828c-22a9-4793-8947-7a36a29db1a8-kube-api-access-4tf4r\") pod \"insights-runtime-extractor-dkrrp\" (UID: \"7774828c-22a9-4793-8947-7a36a29db1a8\") " pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.358446 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.358426 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js9ps\" (UniqueName: \"kubernetes.io/projected/f08d0d3e-3c8d-4eb3-8eef-786babdb96e7-kube-api-access-js9ps\") pod \"ingress-canary-nmczh\" (UID: \"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7\") " pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.358534 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.358516 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7txq\" (UniqueName: \"kubernetes.io/projected/5961e326-2362-41a5-84cb-0eb37fb30150-kube-api-access-f7txq\") pod \"dns-default-pfnk7\" (UID: \"5961e326-2362-41a5-84cb-0eb37fb30150\") " pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.447837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.447799 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:18.452191 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.452170 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nmczh" Jul 10 13:11:18.469531 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.469507 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dkrrp" Jul 10 13:11:18.600545 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.600466 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pfnk7"] Jul 10 13:11:18.604683 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:18.604655 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5961e326_2362_41a5_84cb_0eb37fb30150.slice/crio-10c2d2021eb6ad6692934c92c3900f2c1693335927e836be633a537cd7732ac2 WatchSource:0}: Error finding container 10c2d2021eb6ad6692934c92c3900f2c1693335927e836be633a537cd7732ac2: Status 404 returned error can't find the container with id 10c2d2021eb6ad6692934c92c3900f2c1693335927e836be633a537cd7732ac2 Jul 10 13:11:18.718679 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.718643 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pfnk7" event={"ID":"5961e326-2362-41a5-84cb-0eb37fb30150","Type":"ContainerStarted","Data":"10c2d2021eb6ad6692934c92c3900f2c1693335927e836be633a537cd7732ac2"} Jul 10 13:11:18.818601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.818532 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nmczh"] Jul 10 13:11:18.821500 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:18.821459 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf08d0d3e_3c8d_4eb3_8eef_786babdb96e7.slice/crio-55df0a416f88d447e829439a2b6566ad9a7281c227cc31615ed4f929dab5a27f WatchSource:0}: Error finding container 55df0a416f88d447e829439a2b6566ad9a7281c227cc31615ed4f929dab5a27f: Status 404 returned error can't find the container with id 55df0a416f88d447e829439a2b6566ad9a7281c227cc31615ed4f929dab5a27f Jul 10 13:11:18.821916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:18.821897 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dkrrp"] Jul 10 13:11:18.825215 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:18.825195 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7774828c_22a9_4793_8947_7a36a29db1a8.slice/crio-a58e1e75231514d972b981df59e6b48b196cef6fbf0112ed11c329f815603576 WatchSource:0}: Error finding container a58e1e75231514d972b981df59e6b48b196cef6fbf0112ed11c329f815603576: Status 404 returned error can't find the container with id a58e1e75231514d972b981df59e6b48b196cef6fbf0112ed11c329f815603576 Jul 10 13:11:19.504741 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.504602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:19.504936 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.504865 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:19.505640 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.505152 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:19.508649 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.507776 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 13:11:19.508649 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.508085 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 13:11:19.508649 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.508581 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6xcvz\"" Jul 10 13:11:19.508848 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.508823 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 13:11:19.511155 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.509016 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 13:11:19.511155 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.509253 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnrp9\"" Jul 10 13:11:19.725465 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.725428 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkrrp" event={"ID":"7774828c-22a9-4793-8947-7a36a29db1a8","Type":"ContainerStarted","Data":"a58e1e75231514d972b981df59e6b48b196cef6fbf0112ed11c329f815603576"} Jul 10 13:11:19.727457 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:19.727425 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nmczh" event={"ID":"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7","Type":"ContainerStarted","Data":"55df0a416f88d447e829439a2b6566ad9a7281c227cc31615ed4f929dab5a27f"} Jul 10 13:11:21.763195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.763168 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d7cf7d56c-7tm4j"] Jul 10 13:11:21.862198 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.862165 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7cf7d56c-7tm4j"] Jul 10 13:11:21.862385 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.862252 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.872385 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.872363 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 13:11:21.872522 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.872461 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 13:11:21.881398 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.881375 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vszsp\"" Jul 10 13:11:21.881513 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.881410 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 13:11:21.881584 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.881542 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 13:11:21.881584 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.881576 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 13:11:21.881688 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.881634 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 13:11:21.881894 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.881877 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 13:11:21.887683 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.887662 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 13:11:21.981009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.980975 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-serving-cert\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.981196 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.981036 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-trusted-ca-bundle\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.981196 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.981124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5t84\" (UniqueName: \"kubernetes.io/projected/79d6f996-d23e-4496-9559-2845a9365119-kube-api-access-z5t84\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.981300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.981190 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-oauth-config\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.981300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.981237 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-service-ca\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.981372 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.981299 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-oauth-serving-cert\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:21.981372 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:21.981348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-console-config\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082610 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-service-ca\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082610 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082582 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-oauth-serving-cert\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082610 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082606 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-console-config\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082868 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082624 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-serving-cert\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082868 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082651 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-trusted-ca-bundle\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082868 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5t84\" (UniqueName: \"kubernetes.io/projected/79d6f996-d23e-4496-9559-2845a9365119-kube-api-access-z5t84\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.082868 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.082711 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-oauth-config\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.083346 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.083314 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-console-config\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.083451 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.083424 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-service-ca\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.083665 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.083646 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-oauth-serving-cert\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.083789 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.083767 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-trusted-ca-bundle\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.085268 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.085245 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-serving-cert\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.085550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.085534 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-oauth-config\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.096353 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.096331 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5t84\" (UniqueName: \"kubernetes.io/projected/79d6f996-d23e-4496-9559-2845a9365119-kube-api-access-z5t84\") pod \"console-d7cf7d56c-7tm4j\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.171830 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.171801 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:22.665516 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:22.665490 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b44jh" Jul 10 13:11:23.947991 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:23.947556 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7cf7d56c-7tm4j"] Jul 10 13:11:23.951191 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:23.951153 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d6f996_d23e_4496_9559_2845a9365119.slice/crio-bbaefe74370c3bf94329d0c1df49ff4a5b0c148350511e97ba148dbd98d8fa93 WatchSource:0}: Error finding container bbaefe74370c3bf94329d0c1df49ff4a5b0c148350511e97ba148dbd98d8fa93: Status 404 returned error can't find the container with id bbaefe74370c3bf94329d0c1df49ff4a5b0c148350511e97ba148dbd98d8fa93 Jul 10 13:11:24.743802 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.743770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pfnk7" event={"ID":"5961e326-2362-41a5-84cb-0eb37fb30150","Type":"ContainerStarted","Data":"16ec65a906bd3c413b09598d6b13b4817f3dff832c492c1012c8e461721fbb1a"} Jul 10 13:11:24.743974 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.743831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pfnk7" event={"ID":"5961e326-2362-41a5-84cb-0eb37fb30150","Type":"ContainerStarted","Data":"f6bbc47097e6b9d34b6586237ce4dd6ad97b7a554b4252a5f9b64aa2980a1776"} Jul 10 13:11:24.743974 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.743886 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:24.745610 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.745587 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkrrp" event={"ID":"7774828c-22a9-4793-8947-7a36a29db1a8","Type":"ContainerStarted","Data":"fe2b3771aa53dd5f49110c015cee175bc48d609475847db41953e339b00ac9cb"} Jul 10 13:11:24.745706 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.745636 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkrrp" event={"ID":"7774828c-22a9-4793-8947-7a36a29db1a8","Type":"ContainerStarted","Data":"784e30fdc4dfe16f568f62319e8b832705ad128abb1ab74d84eb7fb7d8f341fe"} Jul 10 13:11:24.746932 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.746913 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nmczh" event={"ID":"f08d0d3e-3c8d-4eb3-8eef-786babdb96e7","Type":"ContainerStarted","Data":"dbbc2cb5c3c79b49a203cf78fdd1b553f5409fec1de97b07613fdb48ddff2d86"} Jul 10 13:11:24.748156 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.748138 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7cf7d56c-7tm4j" event={"ID":"79d6f996-d23e-4496-9559-2845a9365119","Type":"ContainerStarted","Data":"bbaefe74370c3bf94329d0c1df49ff4a5b0c148350511e97ba148dbd98d8fa93"} Jul 10 13:11:24.763320 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.763241 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pfnk7" podStartSLOduration=1.577653623 podStartE2EDuration="6.763211634s" totalImagesPullingTime="5.185558011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:24.761354811 +0000 UTC m=+64.483280693" watchObservedRunningTime="2026-07-10 13:11:24.763211634 +0000 UTC m=+64.485137507" Jul 10 13:11:24.784202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:24.784153 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nmczh" podStartSLOduration=1.812876277 podStartE2EDuration="6.784135365s" totalImagesPullingTime="4.971259088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:24.777641657 +0000 UTC m=+64.499567540" watchObservedRunningTime="2026-07-10 13:11:24.784135365 +0000 UTC m=+64.506061248" Jul 10 13:11:26.848263 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.848231 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-d5dk8"] Jul 10 13:11:26.853992 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.853972 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:26.857351 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.857329 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 13:11:26.858472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.858337 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 13:11:26.858472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.858461 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 13:11:26.858662 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.858481 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-b5msw\"" Jul 10 13:11:26.858662 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.858480 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 13:11:26.858662 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.858564 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 13:11:26.867137 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.867116 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-d5dk8"] Jul 10 13:11:26.918824 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.918783 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:26.918955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.918858 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dj9v\" (UniqueName: \"kubernetes.io/projected/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-kube-api-access-8dj9v\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:26.918955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.918912 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:26.918955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:26.918942 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-metrics-client-ca\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.020024 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.019991 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.020185 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.020037 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dj9v\" (UniqueName: \"kubernetes.io/projected/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-kube-api-access-8dj9v\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.020185 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.020094 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.020185 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.020118 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-metrics-client-ca\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.020342 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:27.020321 2571 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 10 13:11:27.020405 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:11:27.020391 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-tls podName:24f8e8a1-609f-4581-ad06-8cb02f7fbd55 nodeName:}" failed. No retries permitted until 2026-07-10 13:11:27.520370268 +0000 UTC m=+67.242296134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-tls") pod "prometheus-operator-64785b6675-d5dk8" (UID: "24f8e8a1-609f-4581-ad06-8cb02f7fbd55") : secret "prometheus-operator-tls" not found Jul 10 13:11:27.020886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.020850 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-metrics-client-ca\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.022476 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.022452 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.029162 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.029140 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dj9v\" (UniqueName: \"kubernetes.io/projected/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-kube-api-access-8dj9v\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.121422 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.121345 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:27.123757 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.123735 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 13:11:27.134796 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.134772 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04aaf506-cb9a-43ce-9556-3c046193e00b-metrics-certs\") pod \"network-metrics-daemon-cd8lg\" (UID: \"04aaf506-cb9a-43ce-9556-3c046193e00b\") " pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:27.221813 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.221778 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:27.225084 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.225046 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 13:11:27.234601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.234581 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b05fa0c-f2b6-440b-950f-1d316453a43c-original-pull-secret\") pod \"global-pull-secret-syncer-xl7w9\" (UID: \"0b05fa0c-f2b6-440b-950f-1d316453a43c\") " pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:27.322305 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.322267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:27.324577 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.324555 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6xcvz\"" Jul 10 13:11:27.324696 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.324557 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 13:11:27.330595 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.330574 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7w9" Jul 10 13:11:27.332729 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.332696 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cd8lg" Jul 10 13:11:27.335041 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.335023 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 13:11:27.345693 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.345675 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68jdn\" (UniqueName: \"kubernetes.io/projected/31f6fbb3-3804-415f-9755-0837ba776ada-kube-api-access-68jdn\") pod \"network-check-target-mlmb7\" (UID: \"31f6fbb3-3804-415f-9755-0837ba776ada\") " pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:27.524504 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.524474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.526871 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.526850 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24f8e8a1-609f-4581-ad06-8cb02f7fbd55-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-d5dk8\" (UID: \"24f8e8a1-609f-4581-ad06-8cb02f7fbd55\") " pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:27.644143 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.644101 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnrp9\"" Jul 10 13:11:27.651903 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.651875 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:27.764744 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:27.764717 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" Jul 10 13:11:28.779933 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:28.779091 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7cf7d56c-7tm4j" event={"ID":"79d6f996-d23e-4496-9559-2845a9365119","Type":"ContainerStarted","Data":"6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5"} Jul 10 13:11:28.793299 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:28.793261 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xl7w9"] Jul 10 13:11:28.798249 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:28.798217 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b05fa0c_f2b6_440b_950f_1d316453a43c.slice/crio-2b1297ab4617720b4e7883a9504c3d7e08952ab567c1525fd052095c9aac2017 WatchSource:0}: Error finding container 2b1297ab4617720b4e7883a9504c3d7e08952ab567c1525fd052095c9aac2017: Status 404 returned error can't find the container with id 2b1297ab4617720b4e7883a9504c3d7e08952ab567c1525fd052095c9aac2017 Jul 10 13:11:28.802934 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:28.802888 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d7cf7d56c-7tm4j" podStartSLOduration=3.13190683 podStartE2EDuration="7.80287213s" totalImagesPullingTime="4.6709653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:28.80173776 +0000 UTC m=+68.523664063" watchObservedRunningTime="2026-07-10 13:11:28.80287213 +0000 UTC m=+68.524798012" Jul 10 13:11:28.806686 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:28.806661 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cd8lg"] Jul 10 13:11:28.809435 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:28.809411 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04aaf506_cb9a_43ce_9556_3c046193e00b.slice/crio-0e61ee6d204e51c309aae47213c71a0f81730206a1d78eb441b53e1bae073ccf WatchSource:0}: Error finding container 0e61ee6d204e51c309aae47213c71a0f81730206a1d78eb441b53e1bae073ccf: Status 404 returned error can't find the container with id 0e61ee6d204e51c309aae47213c71a0f81730206a1d78eb441b53e1bae073ccf Jul 10 13:11:28.835242 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:28.835192 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-d5dk8"] Jul 10 13:11:28.837228 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:28.837207 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f8e8a1_609f_4581_ad06_8cb02f7fbd55.slice/crio-5ca7fdd7eecb0e5a1a32de6bd6c0b1835c0c4e1e62b154c08d0d3b3985a35f83 WatchSource:0}: Error finding container 5ca7fdd7eecb0e5a1a32de6bd6c0b1835c0c4e1e62b154c08d0d3b3985a35f83: Status 404 returned error can't find the container with id 5ca7fdd7eecb0e5a1a32de6bd6c0b1835c0c4e1e62b154c08d0d3b3985a35f83 Jul 10 13:11:29.032452 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.032407 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mlmb7"] Jul 10 13:11:29.035743 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:29.035713 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f6fbb3_3804_415f_9755_0837ba776ada.slice/crio-40a9b30f40979e89c1b0a2cd8b886b220db849fb89289ccb56bc82f37c0b6392 WatchSource:0}: Error finding container 40a9b30f40979e89c1b0a2cd8b886b220db849fb89289ccb56bc82f37c0b6392: Status 404 returned error can't find the container with id 40a9b30f40979e89c1b0a2cd8b886b220db849fb89289ccb56bc82f37c0b6392 Jul 10 13:11:29.785543 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.785447 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" event={"ID":"24f8e8a1-609f-4581-ad06-8cb02f7fbd55","Type":"ContainerStarted","Data":"5ca7fdd7eecb0e5a1a32de6bd6c0b1835c0c4e1e62b154c08d0d3b3985a35f83"} Jul 10 13:11:29.791212 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.791031 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkrrp" event={"ID":"7774828c-22a9-4793-8947-7a36a29db1a8","Type":"ContainerStarted","Data":"a5adef47e5added240e004b935c3bfaca7debc24d919393696f2391da969402e"} Jul 10 13:11:29.793202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.793143 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mlmb7" event={"ID":"31f6fbb3-3804-415f-9755-0837ba776ada","Type":"ContainerStarted","Data":"40a9b30f40979e89c1b0a2cd8b886b220db849fb89289ccb56bc82f37c0b6392"} Jul 10 13:11:29.794444 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.794408 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xl7w9" event={"ID":"0b05fa0c-f2b6-440b-950f-1d316453a43c","Type":"ContainerStarted","Data":"2b1297ab4617720b4e7883a9504c3d7e08952ab567c1525fd052095c9aac2017"} Jul 10 13:11:29.796439 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.796418 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cd8lg" event={"ID":"04aaf506-cb9a-43ce-9556-3c046193e00b","Type":"ContainerStarted","Data":"0e61ee6d204e51c309aae47213c71a0f81730206a1d78eb441b53e1bae073ccf"} Jul 10 13:11:29.815394 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:29.815355 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dkrrp" podStartSLOduration=2.132962014 podStartE2EDuration="11.815331791s" totalImagesPullingTime="9.682369777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:11:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:29.814384074 +0000 UTC m=+69.536309957" watchObservedRunningTime="2026-07-10 13:11:29.815331791 +0000 UTC m=+69.537257674" Jul 10 13:11:30.803131 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:30.803092 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cd8lg" event={"ID":"04aaf506-cb9a-43ce-9556-3c046193e00b","Type":"ContainerStarted","Data":"648b93829ed355e22fde3f3490e6782596fc6b06f8210ff4bea0731b5e7ff601"} Jul 10 13:11:30.803577 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:30.803162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cd8lg" event={"ID":"04aaf506-cb9a-43ce-9556-3c046193e00b","Type":"ContainerStarted","Data":"e2a13acc950ec458ecce049256b7d89f4d475c53247b6540770424cd8c5def1a"} Jul 10 13:11:30.825256 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:30.825203 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cd8lg" podStartSLOduration=68.553395171 podStartE2EDuration="1m9.825183398s" totalImagesPullingTime="1.271788227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:30.822896998 +0000 UTC m=+70.544822882" watchObservedRunningTime="2026-07-10 13:11:30.825183398 +0000 UTC m=+70.547109281" Jul 10 13:11:31.808406 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:31.808359 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" event={"ID":"24f8e8a1-609f-4581-ad06-8cb02f7fbd55","Type":"ContainerStarted","Data":"8c5360d6857744eecce036743d159b4cfa051843ca4a9b6d04ea4aafb001b844"} Jul 10 13:11:31.808837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:31.808422 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" event={"ID":"24f8e8a1-609f-4581-ad06-8cb02f7fbd55","Type":"ContainerStarted","Data":"27208307860d455c52a3beae3506e9fc9eba9f4b37488d34a60ee1a837beb745"} Jul 10 13:11:32.172472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:32.172377 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:32.172472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:32.172448 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:11:32.174270 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:32.174234 2571 patch_prober.go:28] interesting pod/console-d7cf7d56c-7tm4j container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 10 13:11:32.174402 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:32.174292 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d7cf7d56c-7tm4j" podUID="79d6f996-d23e-4496-9559-2845a9365119" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 10 13:11:33.816369 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:33.816339 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mlmb7" event={"ID":"31f6fbb3-3804-415f-9755-0837ba776ada","Type":"ContainerStarted","Data":"8aea3794ed1b0d06652f7acf1ff31600498e733da1012c4365640884b5862e16"} Jul 10 13:11:33.816822 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:33.816459 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:11:33.817857 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:33.817829 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xl7w9" event={"ID":"0b05fa0c-f2b6-440b-950f-1d316453a43c","Type":"ContainerStarted","Data":"1d0b697b99687d70a3fec87f7b6aade931b0cb0649b63fab0dbcf1d3cc0a45f8"} Jul 10 13:11:33.847171 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:33.847100 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mlmb7" podStartSLOduration=68.291643711 podStartE2EDuration="1m12.847085017s" totalImagesPullingTime="4.555441306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:33.846678772 +0000 UTC m=+73.568604654" watchObservedRunningTime="2026-07-10 13:11:33.847085017 +0000 UTC m=+73.569010899" Jul 10 13:11:33.847319 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:33.847299 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-d5dk8" podStartSLOduration=5.8447583739999995 podStartE2EDuration="7.847294891s" totalImagesPullingTime="2.002536517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:31.840288816 +0000 UTC m=+71.562214698" watchObservedRunningTime="2026-07-10 13:11:33.847294891 +0000 UTC m=+73.569220773" Jul 10 13:11:33.877190 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:33.877145 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xl7w9" podStartSLOduration=68.081454908 podStartE2EDuration="1m12.877129989s" totalImagesPullingTime="4.795675081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:33.876413785 +0000 UTC m=+73.598339668" watchObservedRunningTime="2026-07-10 13:11:33.877129989 +0000 UTC m=+73.599055871" Jul 10 13:11:34.664333 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.664301 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv"] Jul 10 13:11:34.666631 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.666615 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.669487 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.669457 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 13:11:34.669723 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.669710 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qbzkb\"" Jul 10 13:11:34.670527 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.670513 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 13:11:34.689541 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.689520 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv"] Jul 10 13:11:34.754207 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.754184 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pfnk7" Jul 10 13:11:34.769545 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.769523 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zmtdl"] Jul 10 13:11:34.771755 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.771741 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.779247 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.779229 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654c2dad-f0a1-4445-bf21-fc93d5940827-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.779352 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.779263 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzzjs\" (UniqueName: \"kubernetes.io/projected/654c2dad-f0a1-4445-bf21-fc93d5940827-kube-api-access-dzzjs\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.779352 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.779282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654c2dad-f0a1-4445-bf21-fc93d5940827-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.779352 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.779322 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654c2dad-f0a1-4445-bf21-fc93d5940827-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.787465 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.787447 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 13:11:34.787669 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.787653 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t6hvq\"" Jul 10 13:11:34.787749 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.787683 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 13:11:34.787949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.787935 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 13:11:34.880485 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880454 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-textfile\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880505 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzzjs\" (UniqueName: \"kubernetes.io/projected/654c2dad-f0a1-4445-bf21-fc93d5940827-kube-api-access-dzzjs\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880524 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28bv7\" (UniqueName: \"kubernetes.io/projected/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-kube-api-access-28bv7\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880583 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654c2dad-f0a1-4445-bf21-fc93d5940827-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-metrics-client-ca\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880683 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-root\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880698 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-wtmp\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880723 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880748 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-tls\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880799 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654c2dad-f0a1-4445-bf21-fc93d5940827-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880845 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-sys\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.880908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880877 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654c2dad-f0a1-4445-bf21-fc93d5940827-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.881467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.880941 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.881526 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.881481 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654c2dad-f0a1-4445-bf21-fc93d5940827-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.883038 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.883013 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654c2dad-f0a1-4445-bf21-fc93d5940827-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.883158 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.883038 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654c2dad-f0a1-4445-bf21-fc93d5940827-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.940570 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.940514 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzzjs\" (UniqueName: \"kubernetes.io/projected/654c2dad-f0a1-4445-bf21-fc93d5940827-kube-api-access-dzzjs\") pod \"openshift-state-metrics-84bd68b896-tqzjv\" (UID: \"654c2dad-f0a1-4445-bf21-fc93d5940827\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.975480 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.975463 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" Jul 10 13:11:34.981884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.981859 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-sys\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.981976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.981890 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.981976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.981912 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-textfile\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.981976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.981957 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28bv7\" (UniqueName: \"kubernetes.io/projected/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-kube-api-access-28bv7\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.981976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.981961 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-sys\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982002 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-metrics-client-ca\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982048 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-root\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982093 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-wtmp\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982121 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982145 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-tls\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982394 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982163 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-root\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982394 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982298 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-wtmp\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982495 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982472 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-textfile\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.982690 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.982669 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.984070 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.984037 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-tls\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.984479 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.984407 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-metrics-client-ca\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:34.984630 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:34.984611 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:35.023043 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.023016 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28bv7\" (UniqueName: \"kubernetes.io/projected/32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c-kube-api-access-28bv7\") pod \"node-exporter-zmtdl\" (UID: \"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c\") " pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:35.080443 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.080415 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zmtdl" Jul 10 13:11:35.090802 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:35.090777 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32176f3e_0c0d_4c44_8c8c_1ab317b9bf1c.slice/crio-d5fc696460b0dd5a5fa7005fa26143c93080c8b9e05034445254ec5aea38336e WatchSource:0}: Error finding container d5fc696460b0dd5a5fa7005fa26143c93080c8b9e05034445254ec5aea38336e: Status 404 returned error can't find the container with id d5fc696460b0dd5a5fa7005fa26143c93080c8b9e05034445254ec5aea38336e Jul 10 13:11:35.171965 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:35.171941 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654c2dad_f0a1_4445_bf21_fc93d5940827.slice/crio-800c13f3478dd3e590384b0ecf91425c6911270f9a3d7b508a77ea5e6ebcefdd WatchSource:0}: Error finding container 800c13f3478dd3e590384b0ecf91425c6911270f9a3d7b508a77ea5e6ebcefdd: Status 404 returned error can't find the container with id 800c13f3478dd3e590384b0ecf91425c6911270f9a3d7b508a77ea5e6ebcefdd Jul 10 13:11:35.183261 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.183241 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv"] Jul 10 13:11:35.827702 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.827667 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmtdl" event={"ID":"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c","Type":"ContainerStarted","Data":"d5fc696460b0dd5a5fa7005fa26143c93080c8b9e05034445254ec5aea38336e"} Jul 10 13:11:35.829809 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.829781 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" event={"ID":"654c2dad-f0a1-4445-bf21-fc93d5940827","Type":"ContainerStarted","Data":"0d7cb4e2d1c064a12737880a7b1c585afe8e10a4dc0bba98c12097c8ee4f9e4f"} Jul 10 13:11:35.829924 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.829844 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" event={"ID":"654c2dad-f0a1-4445-bf21-fc93d5940827","Type":"ContainerStarted","Data":"5c9931eafc3bf13050a7f989d12ea965396a1ce503b802adc457fe60bbc27a2f"} Jul 10 13:11:35.829924 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:35.829875 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" event={"ID":"654c2dad-f0a1-4445-bf21-fc93d5940827","Type":"ContainerStarted","Data":"800c13f3478dd3e590384b0ecf91425c6911270f9a3d7b508a77ea5e6ebcefdd"} Jul 10 13:11:36.040043 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.040011 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 13:11:36.042869 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.042847 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.052429 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.052405 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 13:11:36.052629 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.052598 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wdw2d\"" Jul 10 13:11:36.053681 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.053663 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 13:11:36.053810 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.053788 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 13:11:36.054318 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.054284 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 13:11:36.054434 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.054330 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 13:11:36.054434 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.054354 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 13:11:36.056133 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.056116 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 13:11:36.057569 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.057543 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 13:11:36.058275 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.058263 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 13:11:36.090583 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090533 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e15db714-8bcb-4279-b35a-b9c11f7da3fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090583 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090561 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl2rp\" (UniqueName: \"kubernetes.io/projected/e15db714-8bcb-4279-b35a-b9c11f7da3fb-kube-api-access-zl2rp\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090723 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090723 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090611 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e15db714-8bcb-4279-b35a-b9c11f7da3fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090723 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090674 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e15db714-8bcb-4279-b35a-b9c11f7da3fb-config-out\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090846 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090821 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090891 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090865 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e15db714-8bcb-4279-b35a-b9c11f7da3fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090942 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090893 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15db714-8bcb-4279-b35a-b9c11f7da3fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.090942 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.091045 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090953 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-web-config\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.091045 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.090995 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.091168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.091108 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.091168 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.091147 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.098462 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.098437 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 13:11:36.191891 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.191856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e15db714-8bcb-4279-b35a-b9c11f7da3fb-config-out\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192071 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.191938 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192071 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.191978 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e15db714-8bcb-4279-b35a-b9c11f7da3fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192071 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192009 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15db714-8bcb-4279-b35a-b9c11f7da3fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192071 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192038 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-web-config\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192111 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192143 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192177 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192206 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e15db714-8bcb-4279-b35a-b9c11f7da3fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192229 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl2rp\" (UniqueName: \"kubernetes.io/projected/e15db714-8bcb-4279-b35a-b9c11f7da3fb-kube-api-access-zl2rp\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192254 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.192302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.192281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e15db714-8bcb-4279-b35a-b9c11f7da3fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.193593 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.193012 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e15db714-8bcb-4279-b35a-b9c11f7da3fb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.193593 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.193311 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e15db714-8bcb-4279-b35a-b9c11f7da3fb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.193593 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.193478 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15db714-8bcb-4279-b35a-b9c11f7da3fb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.196885 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.196325 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.197523 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.197462 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-web-config\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.197693 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.197658 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.200360 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.200274 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e15db714-8bcb-4279-b35a-b9c11f7da3fb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.200485 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.200463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.200632 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.200564 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.201040 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.201020 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e15db714-8bcb-4279-b35a-b9c11f7da3fb-config-out\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.202025 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.201974 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.203613 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.203587 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e15db714-8bcb-4279-b35a-b9c11f7da3fb-config-volume\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.222403 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.222379 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl2rp\" (UniqueName: \"kubernetes.io/projected/e15db714-8bcb-4279-b35a-b9c11f7da3fb-kube-api-access-zl2rp\") pod \"alertmanager-main-0\" (UID: \"e15db714-8bcb-4279-b35a-b9c11f7da3fb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.353877 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.353812 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 13:11:36.541778 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.541753 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 13:11:36.627988 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:11:36.627913 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15db714_8bcb_4279_b35a_b9c11f7da3fb.slice/crio-0c29d5b573a788d02ae31857af8638f733c0f14c5ddd17a0b90ad433429b56c8 WatchSource:0}: Error finding container 0c29d5b573a788d02ae31857af8638f733c0f14c5ddd17a0b90ad433429b56c8: Status 404 returned error can't find the container with id 0c29d5b573a788d02ae31857af8638f733c0f14c5ddd17a0b90ad433429b56c8 Jul 10 13:11:36.834118 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.834085 2571 generic.go:356] "Generic (PLEG): container finished" podID="32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c" containerID="7af10d16a0ecbe7137832905ffa46ba9bd7fbc6c209668bcf5a48b8ee5ab0b1a" exitCode=0 Jul 10 13:11:36.834246 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.834176 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmtdl" event={"ID":"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c","Type":"ContainerDied","Data":"7af10d16a0ecbe7137832905ffa46ba9bd7fbc6c209668bcf5a48b8ee5ab0b1a"} Jul 10 13:11:36.836342 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.836316 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" event={"ID":"654c2dad-f0a1-4445-bf21-fc93d5940827","Type":"ContainerStarted","Data":"fd568250524dae9a590e9c6d166aaa13078137ccd114da1727b741351d7712c7"} Jul 10 13:11:36.837598 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.837579 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"0c29d5b573a788d02ae31857af8638f733c0f14c5ddd17a0b90ad433429b56c8"} Jul 10 13:11:36.904881 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:36.904830 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tqzjv" podStartSLOduration=1.526485671 podStartE2EDuration="2.904810346s" totalImagesPullingTime="1.378324675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:36.904356141 +0000 UTC m=+76.626282020" watchObservedRunningTime="2026-07-10 13:11:36.904810346 +0000 UTC m=+76.626736229" Jul 10 13:11:37.845036 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:37.844985 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmtdl" event={"ID":"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c","Type":"ContainerStarted","Data":"0e2f931fbf723c5dc477d1af625d832e3f0d480404cbc03392a23edabdcec019"} Jul 10 13:11:37.845484 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:37.845044 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmtdl" event={"ID":"32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c","Type":"ContainerStarted","Data":"b5f932125feee5ca98c581a865f0b748c475aadf0dce247c2da4d8bdb8a97f2a"} Jul 10 13:11:37.884779 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:37.884718 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zmtdl" podStartSLOduration=2.280360827 podStartE2EDuration="3.884698557s" totalImagesPullingTime="1.539586191s" totalInitContainerRuntime="64.751539ms" isStatefulPod=true podCreationTimestamp="2026-07-10 13:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:37.88439402 +0000 UTC m=+77.606319903" watchObservedRunningTime="2026-07-10 13:11:37.884698557 +0000 UTC m=+77.606624440" Jul 10 13:11:38.849384 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:38.849348 2571 generic.go:356] "Generic (PLEG): container finished" podID="e15db714-8bcb-4279-b35a-b9c11f7da3fb" containerID="12e1784d4601d3e40e8f0116c0a805193c606d0f3dd8f4652af9d9146f4d7485" exitCode=0 Jul 10 13:11:38.849853 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:38.849439 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerDied","Data":"12e1784d4601d3e40e8f0116c0a805193c606d0f3dd8f4652af9d9146f4d7485"} Jul 10 13:11:40.690509 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:40.690476 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d7cf7d56c-7tm4j"] Jul 10 13:11:41.348587 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.348073 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 13:11:41.351442 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.351417 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.362302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.362276 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 13:11:41.364980 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.364964 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 13:11:41.365226 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365206 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 13:11:41.365334 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365231 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vh8nc\"" Jul 10 13:11:41.365334 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365282 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 13:11:41.365449 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365404 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 13:11:41.365500 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365483 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 13:11:41.365555 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365501 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 13:11:41.365744 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365728 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 13:11:41.365744 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.365739 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 13:11:41.366436 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.366263 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 13:11:41.368981 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.368963 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 13:11:41.369846 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.369828 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 13:11:41.371267 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.371251 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 13:11:41.371340 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.371268 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 13:11:41.374019 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.374003 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-flllprtrgh10o\"" Jul 10 13:11:41.375977 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.375961 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 13:11:41.392087 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.392043 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429679 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429733 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429771 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429805 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429842 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429878 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429906 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-web-config\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429937 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429961 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.429985 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430010 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddc2a65e-6b13-4bba-880b-ac49b46b7433-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430080 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430104 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430134 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.430766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430162 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-config\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.431343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430197 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfsb4\" (UniqueName: \"kubernetes.io/projected/ddc2a65e-6b13-4bba-880b-ac49b46b7433-kube-api-access-kfsb4\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.431343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430231 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.431343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430278 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.431343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.430303 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddc2a65e-6b13-4bba-880b-ac49b46b7433-config-out\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.530634 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530612 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddc2a65e-6b13-4bba-880b-ac49b46b7433-config-out\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.530775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530642 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.530775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530665 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.530775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530691 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.530775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.530775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530741 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530809 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-web-config\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530840 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530866 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530888 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddc2a65e-6b13-4bba-880b-ac49b46b7433-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530956 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.530982 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531032 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531013 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531044 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-config\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531093 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfsb4\" (UniqueName: \"kubernetes.io/projected/ddc2a65e-6b13-4bba-880b-ac49b46b7433-kube-api-access-kfsb4\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531721 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531704 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.531769 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531750 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.532324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.532298 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.532463 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.531747 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.532524 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.532464 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.533755 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.533440 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddc2a65e-6b13-4bba-880b-ac49b46b7433-config-out\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.534152 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.534128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.534253 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.534209 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddc2a65e-6b13-4bba-880b-ac49b46b7433-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.534440 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.534413 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.534748 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.534718 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ddc2a65e-6b13-4bba-880b-ac49b46b7433-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.534856 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.534829 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddc2a65e-6b13-4bba-880b-ac49b46b7433-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.535231 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.535204 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.535415 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.535396 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.535953 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.535926 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-config\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.536349 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.536304 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.536563 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.536535 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-web-config\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.538955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.537813 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.538955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.538672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.539566 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.539180 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ddc2a65e-6b13-4bba-880b-ac49b46b7433-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.554007 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.553987 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfsb4\" (UniqueName: \"kubernetes.io/projected/ddc2a65e-6b13-4bba-880b-ac49b46b7433-kube-api-access-kfsb4\") pod \"prometheus-k8s-0\" (UID: \"ddc2a65e-6b13-4bba-880b-ac49b46b7433\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.661319 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.661287 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:11:41.841791 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.841764 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 13:11:41.869202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.869173 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"1e3b6fbc11bb66fd9a52e7aa7f6beb2d470f83a02910ac015f8cdc92793c1618"} Jul 10 13:11:41.869408 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.869390 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"03a73b877cb6830265b19ef00860328dba2d7b0931e685bf5bc428e305e5f02b"} Jul 10 13:11:41.869515 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.869497 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"5c5845decb0a9a3b65b1b544ad5ea726ae59475741cc5f108d952d4fe8636d8f"} Jul 10 13:11:41.869587 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.869547 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"aec9b59e678ec91c59f539c3a252de2648337182ddb80d98307cf86ea77d546e"} Jul 10 13:11:41.869642 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.869593 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"eac396f0c869fc33f54e3b9829a1f935fa37acbf7c0c778f6b7d60bb05a5a97b"} Jul 10 13:11:41.870573 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:41.870548 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"7f060a196e7a3a08d63b113d3f54ec7f30ddb28200f82cd8caa3ed5ddde3001f"} Jul 10 13:11:42.878643 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:42.878609 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e15db714-8bcb-4279-b35a-b9c11f7da3fb","Type":"ContainerStarted","Data":"b3e8585c4969addc6b2719055c0a54182e0756cbf4515b282a330c055937d785"} Jul 10 13:11:42.880085 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:42.880045 2571 generic.go:356] "Generic (PLEG): container finished" podID="ddc2a65e-6b13-4bba-880b-ac49b46b7433" containerID="46e34debb5c63937d0dccc913036452751633739cecdb4dc05a3a2163e2689fd" exitCode=0 Jul 10 13:11:42.880085 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:42.880077 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerDied","Data":"46e34debb5c63937d0dccc913036452751633739cecdb4dc05a3a2163e2689fd"} Jul 10 13:11:42.917266 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:42.917225 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.561941879 podStartE2EDuration="7.917209796s" totalImagesPullingTime="4.341114942s" totalInitContainerRuntime="14.152975ms" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:42.915586092 +0000 UTC m=+82.637511996" watchObservedRunningTime="2026-07-10 13:11:42.917209796 +0000 UTC m=+82.639135678" Jul 10 13:11:47.900220 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:47.900182 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"3d0f3c78dff291b315201e8b110ddf6f5e10300793a3b8b4b0f76fd04774f793"} Jul 10 13:11:47.900569 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:47.900261 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"18a0746550d9f3dc79d54a4b0863f39fea8d734c89dfd37aefccad71cc2f28ff"} Jul 10 13:11:50.916656 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:50.916623 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"74c5cbb5caaedb4695690b87fd8b2672975d8fbc0b7ce3ea9706ad21b7af49ee"} Jul 10 13:11:50.917016 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:50.916687 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"8dbfe190b7627ee754850ed8a761ab217360bf8cf63a44e766c9d22dcc8a0eeb"} Jul 10 13:11:50.917016 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:50.916714 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"7a13cf538446a0b110f112db7169ea04e6806ca96d64fd1403156c058566b129"} Jul 10 13:11:50.917016 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:50.916737 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddc2a65e-6b13-4bba-880b-ac49b46b7433","Type":"ContainerStarted","Data":"4d10f2699b5e68e1a2238243dec78a5987c5281eca69c31dd7a7e482a385bf24"} Jul 10 13:11:50.958077 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:50.958012 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.894836506 podStartE2EDuration="9.957992401s" totalImagesPullingTime="7.042597405s" totalInitContainerRuntime="20.55849ms" isStatefulPod=false podCreationTimestamp="2026-07-10 13:11:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:11:50.955840963 +0000 UTC m=+90.677766836" watchObservedRunningTime="2026-07-10 13:11:50.957992401 +0000 UTC m=+90.679918285" Jul 10 13:11:51.662204 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:11:51.662176 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:12:04.823420 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:04.823390 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mlmb7" Jul 10 13:12:05.710813 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:05.710770 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d7cf7d56c-7tm4j" podUID="79d6f996-d23e-4496-9559-2845a9365119" containerName="console" containerID="cri-o://6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5" gracePeriod=15 Jul 10 13:12:06.482570 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.482547 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:12:06.634594 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634530 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5t84\" (UniqueName: \"kubernetes.io/projected/79d6f996-d23e-4496-9559-2845a9365119-kube-api-access-z5t84\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.634594 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634557 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-oauth-config\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.634594 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634583 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-serving-cert\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.634789 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634615 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-console-config\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.634789 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634677 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-oauth-serving-cert\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.634789 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634723 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-service-ca\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.634789 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.634750 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-trusted-ca-bundle\") pod \"79d6f996-d23e-4496-9559-2845a9365119\" (UID: \"79d6f996-d23e-4496-9559-2845a9365119\") " Jul 10 13:12:06.635199 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.635166 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-service-ca" (OuterVolumeSpecName: "service-ca") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:12:06.635332 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.635251 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-console-config" (OuterVolumeSpecName: "console-config") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:12:06.635332 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.635246 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:12:06.635492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.635465 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:12:06.637030 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.637005 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:12:06.637030 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.637023 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79d6f996-d23e-4496-9559-2845a9365119-kube-api-access-z5t84" (OuterVolumeSpecName: "kube-api-access-z5t84") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "kube-api-access-z5t84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:12:06.637192 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.637041 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "79d6f996-d23e-4496-9559-2845a9365119" (UID: "79d6f996-d23e-4496-9559-2845a9365119"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:12:06.736265 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736226 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5t84\" (UniqueName: \"kubernetes.io/projected/79d6f996-d23e-4496-9559-2845a9365119-kube-api-access-z5t84\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.736265 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736259 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-oauth-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.736265 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736269 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79d6f996-d23e-4496-9559-2845a9365119-console-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.736265 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736278 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-console-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.736494 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736286 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-oauth-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.736494 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736294 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.736494 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.736304 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79d6f996-d23e-4496-9559-2845a9365119-trusted-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:12:06.975733 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.975666 2571 generic.go:356] "Generic (PLEG): container finished" podID="79d6f996-d23e-4496-9559-2845a9365119" containerID="6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5" exitCode=0 Jul 10 13:12:06.975733 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.975715 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7cf7d56c-7tm4j" Jul 10 13:12:06.975888 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.975754 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7cf7d56c-7tm4j" event={"ID":"79d6f996-d23e-4496-9559-2845a9365119","Type":"ContainerDied","Data":"6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5"} Jul 10 13:12:06.975888 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.975808 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7cf7d56c-7tm4j" event={"ID":"79d6f996-d23e-4496-9559-2845a9365119","Type":"ContainerDied","Data":"bbaefe74370c3bf94329d0c1df49ff4a5b0c148350511e97ba148dbd98d8fa93"} Jul 10 13:12:06.975888 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.975828 2571 scope.go:117] "RemoveContainer" containerID="6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5" Jul 10 13:12:06.995110 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.995081 2571 scope.go:117] "RemoveContainer" containerID="6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5" Jul 10 13:12:06.995378 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:12:06.995354 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5\": container with ID starting with 6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5 not found: ID does not exist" containerID="6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5" Jul 10 13:12:06.995460 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.995393 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5"} err="failed to get container status \"6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5\": rpc error: code = NotFound desc = could not find container \"6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5\": container with ID starting with 6035d315a3803a39db018f472a09e56619237bb8a13aacd48eb074ba0aab6ae5 not found: ID does not exist" Jul 10 13:12:06.997676 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.997657 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d7cf7d56c-7tm4j"] Jul 10 13:12:06.999449 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:06.999425 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d7cf7d56c-7tm4j"] Jul 10 13:12:07.508667 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:07.508633 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79d6f996-d23e-4496-9559-2845a9365119" path="/var/lib/kubelet/pods/79d6f996-d23e-4496-9559-2845a9365119/volumes" Jul 10 13:12:41.661918 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:41.661891 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:12:41.681912 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:41.681881 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:12:42.115478 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:12:42.115450 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 13:13:05.843368 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.843334 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8556d6f957-xt89g"] Jul 10 13:13:05.843792 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.843608 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79d6f996-d23e-4496-9559-2845a9365119" containerName="console" Jul 10 13:13:05.843792 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.843620 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79d6f996-d23e-4496-9559-2845a9365119" containerName="console" Jul 10 13:13:05.843792 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.843671 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79d6f996-d23e-4496-9559-2845a9365119" containerName="console" Jul 10 13:13:05.846167 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.846143 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.864214 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.864189 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 13:13:05.864324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.864264 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 13:13:05.864324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.864264 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 13:13:05.864519 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.864503 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 13:13:05.864567 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.864540 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 13:13:05.865202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.865186 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vszsp\"" Jul 10 13:13:05.865364 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.865352 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 13:13:05.865423 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.865376 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 13:13:05.885992 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.885965 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 13:13:05.942450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.942428 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8556d6f957-xt89g"] Jul 10 13:13:05.947045 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947021 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvtnc\" (UniqueName: \"kubernetes.io/projected/163c5963-a221-4ef2-b4d4-0de0da86bcaf-kube-api-access-wvtnc\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.947289 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947268 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-oauth-serving-cert\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.947451 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947424 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-config\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.947560 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947546 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-oauth-config\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.947672 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947660 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-serving-cert\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.947761 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947749 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-service-ca\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:05.947862 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:05.947848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-trusted-ca-bundle\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048503 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048472 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-config\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048503 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048507 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-oauth-config\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048532 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-serving-cert\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048547 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-service-ca\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048580 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-trusted-ca-bundle\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048608 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvtnc\" (UniqueName: \"kubernetes.io/projected/163c5963-a221-4ef2-b4d4-0de0da86bcaf-kube-api-access-wvtnc\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.048746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.048652 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-oauth-serving-cert\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.049376 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.049346 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-config\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.049376 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.049371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-service-ca\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.049568 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.049386 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-oauth-serving-cert\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.049568 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.049565 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-trusted-ca-bundle\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.051202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.051178 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-serving-cert\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.051303 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.051178 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-oauth-config\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.063319 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.063296 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvtnc\" (UniqueName: \"kubernetes.io/projected/163c5963-a221-4ef2-b4d4-0de0da86bcaf-kube-api-access-wvtnc\") pod \"console-8556d6f957-xt89g\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.154495 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.154428 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:06.298372 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:06.298344 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8556d6f957-xt89g"] Jul 10 13:13:06.301169 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:13:06.301143 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163c5963_a221_4ef2_b4d4_0de0da86bcaf.slice/crio-e6f34762f6e4fe2b676eb985a4e346b07138c02e336344039f980275eb021959 WatchSource:0}: Error finding container e6f34762f6e4fe2b676eb985a4e346b07138c02e336344039f980275eb021959: Status 404 returned error can't find the container with id e6f34762f6e4fe2b676eb985a4e346b07138c02e336344039f980275eb021959 Jul 10 13:13:07.185420 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:07.185376 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8556d6f957-xt89g" event={"ID":"163c5963-a221-4ef2-b4d4-0de0da86bcaf","Type":"ContainerStarted","Data":"6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f"} Jul 10 13:13:07.185791 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:07.185429 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8556d6f957-xt89g" event={"ID":"163c5963-a221-4ef2-b4d4-0de0da86bcaf","Type":"ContainerStarted","Data":"e6f34762f6e4fe2b676eb985a4e346b07138c02e336344039f980275eb021959"} Jul 10 13:13:07.210188 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:07.210135 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8556d6f957-xt89g" podStartSLOduration=2.210121372 podStartE2EDuration="2.210121372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:13:07.209188951 +0000 UTC m=+166.931114833" watchObservedRunningTime="2026-07-10 13:13:07.210121372 +0000 UTC m=+166.932047254" Jul 10 13:13:16.155608 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.155521 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:16.155608 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.155590 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:16.160379 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.160357 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:16.220331 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.220307 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:16.604096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.604067 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7575c499b6-ttgkq"] Jul 10 13:13:16.606448 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.606424 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.646296 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.646270 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7575c499b6-ttgkq"] Jul 10 13:13:16.728114 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-service-ca\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.728222 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728129 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-oauth-serving-cert\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.728222 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728189 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-trusted-ca-bundle\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.728300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728243 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhs7c\" (UniqueName: \"kubernetes.io/projected/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-kube-api-access-lhs7c\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.728300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728270 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-oauth-config\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.728300 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728291 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-config\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.728393 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.728321 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-serving-cert\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.828870 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.828843 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhs7c\" (UniqueName: \"kubernetes.io/projected/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-kube-api-access-lhs7c\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.828987 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.828875 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-oauth-config\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.828987 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.828900 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-config\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.828987 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.828928 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-serving-cert\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829178 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829039 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-service-ca\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829178 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829134 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-oauth-serving-cert\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829178 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829170 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-trusted-ca-bundle\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829725 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829700 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-config\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829755 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-service-ca\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829825 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-oauth-serving-cert\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.829996 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.829979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-trusted-ca-bundle\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.831252 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.831234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-oauth-config\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.831347 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.831333 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-console-serving-cert\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.841898 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.841877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhs7c\" (UniqueName: \"kubernetes.io/projected/a4a4f81d-ced2-4098-afd4-2f8d84f8f72e-kube-api-access-lhs7c\") pod \"console-7575c499b6-ttgkq\" (UID: \"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e\") " pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:16.915928 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:16.915873 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:17.053903 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:17.053867 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7575c499b6-ttgkq"] Jul 10 13:13:17.059021 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:13:17.058995 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a4f81d_ced2_4098_afd4_2f8d84f8f72e.slice/crio-5dd8b29833491fab72166cbf48ced140ff9eb8b8c50e259bf8c7f23817db5239 WatchSource:0}: Error finding container 5dd8b29833491fab72166cbf48ced140ff9eb8b8c50e259bf8c7f23817db5239: Status 404 returned error can't find the container with id 5dd8b29833491fab72166cbf48ced140ff9eb8b8c50e259bf8c7f23817db5239 Jul 10 13:13:17.221877 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:17.221789 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7575c499b6-ttgkq" event={"ID":"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e","Type":"ContainerStarted","Data":"ea9a982ff132a703a8ea0d8bfe428883027b7c4e0613df8031d2b43e61702365"} Jul 10 13:13:17.221877 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:17.221844 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7575c499b6-ttgkq" event={"ID":"a4a4f81d-ced2-4098-afd4-2f8d84f8f72e","Type":"ContainerStarted","Data":"5dd8b29833491fab72166cbf48ced140ff9eb8b8c50e259bf8c7f23817db5239"} Jul 10 13:13:17.256612 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:17.256566 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7575c499b6-ttgkq" podStartSLOduration=1.256549359 podStartE2EDuration="1.256549359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:13:17.256098706 +0000 UTC m=+176.978024589" watchObservedRunningTime="2026-07-10 13:13:17.256549359 +0000 UTC m=+176.978475275" Jul 10 13:13:26.916560 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:26.916530 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:26.916925 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:26.916580 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:26.921188 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:26.921161 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:27.262322 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:27.262295 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7575c499b6-ttgkq" Jul 10 13:13:27.347584 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:27.347550 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8556d6f957-xt89g"] Jul 10 13:13:52.367398 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.367331 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8556d6f957-xt89g" podUID="163c5963-a221-4ef2-b4d4-0de0da86bcaf" containerName="console" containerID="cri-o://6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f" gracePeriod=15 Jul 10 13:13:52.606450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.606424 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:52.704414 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704327 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-trusted-ca-bundle\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.704414 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704373 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-config\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.704635 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704417 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-serving-cert\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.704635 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704529 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-service-ca\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.704635 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704570 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-oauth-config\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.704635 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704610 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvtnc\" (UniqueName: \"kubernetes.io/projected/163c5963-a221-4ef2-b4d4-0de0da86bcaf-kube-api-access-wvtnc\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.704925 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704873 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:13:52.704925 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704918 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-oauth-serving-cert\") pod \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\" (UID: \"163c5963-a221-4ef2-b4d4-0de0da86bcaf\") " Jul 10 13:13:52.705193 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.704988 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-config" (OuterVolumeSpecName: "console-config") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:13:52.705193 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.705006 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-service-ca" (OuterVolumeSpecName: "service-ca") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:13:52.705318 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.705280 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-trusted-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:52.705318 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.705300 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:52.705318 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.705315 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:52.705467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.705313 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:13:52.706674 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.706649 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/163c5963-a221-4ef2-b4d4-0de0da86bcaf-kube-api-access-wvtnc" (OuterVolumeSpecName: "kube-api-access-wvtnc") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "kube-api-access-wvtnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:13:52.706759 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.706680 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:13:52.706759 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.706711 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "163c5963-a221-4ef2-b4d4-0de0da86bcaf" (UID: "163c5963-a221-4ef2-b4d4-0de0da86bcaf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:13:52.806181 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.806147 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/163c5963-a221-4ef2-b4d4-0de0da86bcaf-oauth-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:52.806181 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.806173 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:52.806181 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.806182 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/163c5963-a221-4ef2-b4d4-0de0da86bcaf-console-oauth-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:52.806181 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:52.806190 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvtnc\" (UniqueName: \"kubernetes.io/projected/163c5963-a221-4ef2-b4d4-0de0da86bcaf-kube-api-access-wvtnc\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:13:53.348591 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.348556 2571 generic.go:356] "Generic (PLEG): container finished" podID="163c5963-a221-4ef2-b4d4-0de0da86bcaf" containerID="6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f" exitCode=0 Jul 10 13:13:53.348747 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.348604 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8556d6f957-xt89g" Jul 10 13:13:53.348747 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.348636 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8556d6f957-xt89g" event={"ID":"163c5963-a221-4ef2-b4d4-0de0da86bcaf","Type":"ContainerDied","Data":"6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f"} Jul 10 13:13:53.348747 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.348688 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8556d6f957-xt89g" event={"ID":"163c5963-a221-4ef2-b4d4-0de0da86bcaf","Type":"ContainerDied","Data":"e6f34762f6e4fe2b676eb985a4e346b07138c02e336344039f980275eb021959"} Jul 10 13:13:53.348747 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.348708 2571 scope.go:117] "RemoveContainer" containerID="6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f" Jul 10 13:13:53.357116 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.357094 2571 scope.go:117] "RemoveContainer" containerID="6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f" Jul 10 13:13:53.357355 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:13:53.357339 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f\": container with ID starting with 6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f not found: ID does not exist" containerID="6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f" Jul 10 13:13:53.357409 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.357362 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f"} err="failed to get container status \"6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f\": rpc error: code = NotFound desc = could not find container \"6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f\": container with ID starting with 6b8c0541376b5b7125f998d7e537813736d29b199756338e5141ff9cb4cc531f not found: ID does not exist" Jul 10 13:13:53.380194 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.380171 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8556d6f957-xt89g"] Jul 10 13:13:53.390573 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.390553 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8556d6f957-xt89g"] Jul 10 13:13:53.506382 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:13:53.506351 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="163c5963-a221-4ef2-b4d4-0de0da86bcaf" path="/var/lib/kubelet/pods/163c5963-a221-4ef2-b4d4-0de0da86bcaf/volumes" Jul 10 13:14:46.579661 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.579584 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5txnr"] Jul 10 13:14:46.580138 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.579857 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163c5963-a221-4ef2-b4d4-0de0da86bcaf" containerName="console" Jul 10 13:14:46.580138 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.579868 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="163c5963-a221-4ef2-b4d4-0de0da86bcaf" containerName="console" Jul 10 13:14:46.580138 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.579917 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="163c5963-a221-4ef2-b4d4-0de0da86bcaf" containerName="console" Jul 10 13:14:46.581990 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.581974 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.584849 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.584818 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 13:14:46.584967 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.584913 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 13:14:46.585897 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.585883 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-nphlq\"" Jul 10 13:14:46.610226 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.610203 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5txnr"] Jul 10 13:14:46.709472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.709431 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm6p2\" (UniqueName: \"kubernetes.io/projected/8804f0c5-f0f7-475d-8421-7ad4afddad37-kube-api-access-fm6p2\") pod \"cert-manager-webhook-6486ffd997-5txnr\" (UID: \"8804f0c5-f0f7-475d-8421-7ad4afddad37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.709472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.709476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8804f0c5-f0f7-475d-8421-7ad4afddad37-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5txnr\" (UID: \"8804f0c5-f0f7-475d-8421-7ad4afddad37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.810478 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.810439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm6p2\" (UniqueName: \"kubernetes.io/projected/8804f0c5-f0f7-475d-8421-7ad4afddad37-kube-api-access-fm6p2\") pod \"cert-manager-webhook-6486ffd997-5txnr\" (UID: \"8804f0c5-f0f7-475d-8421-7ad4afddad37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.810647 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.810490 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8804f0c5-f0f7-475d-8421-7ad4afddad37-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5txnr\" (UID: \"8804f0c5-f0f7-475d-8421-7ad4afddad37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.823992 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.823958 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm6p2\" (UniqueName: \"kubernetes.io/projected/8804f0c5-f0f7-475d-8421-7ad4afddad37-kube-api-access-fm6p2\") pod \"cert-manager-webhook-6486ffd997-5txnr\" (UID: \"8804f0c5-f0f7-475d-8421-7ad4afddad37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.831318 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.831259 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8804f0c5-f0f7-475d-8421-7ad4afddad37-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5txnr\" (UID: \"8804f0c5-f0f7-475d-8421-7ad4afddad37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:46.906198 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:46.906160 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:47.057222 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:47.057189 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5txnr"] Jul 10 13:14:47.060391 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:14:47.060361 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8804f0c5_f0f7_475d_8421_7ad4afddad37.slice/crio-46b57b79bdfd62e5ff1fc5580005b355c5360533fd4ad64ab63972be7143a226 WatchSource:0}: Error finding container 46b57b79bdfd62e5ff1fc5580005b355c5360533fd4ad64ab63972be7143a226: Status 404 returned error can't find the container with id 46b57b79bdfd62e5ff1fc5580005b355c5360533fd4ad64ab63972be7143a226 Jul 10 13:14:47.535475 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:47.535443 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" event={"ID":"8804f0c5-f0f7-475d-8421-7ad4afddad37","Type":"ContainerStarted","Data":"46b57b79bdfd62e5ff1fc5580005b355c5360533fd4ad64ab63972be7143a226"} Jul 10 13:14:50.548759 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:50.548717 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" event={"ID":"8804f0c5-f0f7-475d-8421-7ad4afddad37","Type":"ContainerStarted","Data":"cd674c40d4860a792ddec666d3e7e842f2cd0f88800686289cd864f20cdd2742"} Jul 10 13:14:50.549144 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:50.548790 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:14:50.571316 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:50.571272 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" podStartSLOduration=1.692766265 podStartE2EDuration="4.571256763s" totalImagesPullingTime="2.878490498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:14:50.570309295 +0000 UTC m=+270.292235177" watchObservedRunningTime="2026-07-10 13:14:50.571256763 +0000 UTC m=+270.293182642" Jul 10 13:14:56.554294 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:14:56.554262 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5txnr" Jul 10 13:15:06.785240 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.785205 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2r9v9"] Jul 10 13:15:06.787727 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.787710 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:06.790317 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.790292 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g86ft\"" Jul 10 13:15:06.800200 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.800178 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2r9v9"] Jul 10 13:15:06.859851 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.859822 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d20e932-4815-4a33-b4ec-196558724563-bound-sa-token\") pod \"cert-manager-55b4d954d5-2r9v9\" (UID: \"5d20e932-4815-4a33-b4ec-196558724563\") " pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:06.859993 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.859867 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmvzz\" (UniqueName: \"kubernetes.io/projected/5d20e932-4815-4a33-b4ec-196558724563-kube-api-access-jmvzz\") pod \"cert-manager-55b4d954d5-2r9v9\" (UID: \"5d20e932-4815-4a33-b4ec-196558724563\") " pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:06.961078 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.961025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmvzz\" (UniqueName: \"kubernetes.io/projected/5d20e932-4815-4a33-b4ec-196558724563-kube-api-access-jmvzz\") pod \"cert-manager-55b4d954d5-2r9v9\" (UID: \"5d20e932-4815-4a33-b4ec-196558724563\") " pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:06.961244 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.961115 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d20e932-4815-4a33-b4ec-196558724563-bound-sa-token\") pod \"cert-manager-55b4d954d5-2r9v9\" (UID: \"5d20e932-4815-4a33-b4ec-196558724563\") " pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:06.968807 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.968778 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d20e932-4815-4a33-b4ec-196558724563-bound-sa-token\") pod \"cert-manager-55b4d954d5-2r9v9\" (UID: \"5d20e932-4815-4a33-b4ec-196558724563\") " pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:06.969224 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:06.969202 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmvzz\" (UniqueName: \"kubernetes.io/projected/5d20e932-4815-4a33-b4ec-196558724563-kube-api-access-jmvzz\") pod \"cert-manager-55b4d954d5-2r9v9\" (UID: \"5d20e932-4815-4a33-b4ec-196558724563\") " pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:07.096811 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:07.096732 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2r9v9" Jul 10 13:15:07.220091 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:07.219936 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2r9v9"] Jul 10 13:15:07.222727 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:15:07.222694 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d20e932_4815_4a33_b4ec_196558724563.slice/crio-11e47209a2112b4ecc8312b5cbddcf260d8577546f4a963fceb9334fca7ca56a WatchSource:0}: Error finding container 11e47209a2112b4ecc8312b5cbddcf260d8577546f4a963fceb9334fca7ca56a: Status 404 returned error can't find the container with id 11e47209a2112b4ecc8312b5cbddcf260d8577546f4a963fceb9334fca7ca56a Jul 10 13:15:07.611604 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:07.611568 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2r9v9" event={"ID":"5d20e932-4815-4a33-b4ec-196558724563","Type":"ContainerStarted","Data":"3f2642b649f6e49415716c2e254477021df60a5c0997cbf0afa293edbba1b923"} Jul 10 13:15:07.611604 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:07.611610 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2r9v9" event={"ID":"5d20e932-4815-4a33-b4ec-196558724563","Type":"ContainerStarted","Data":"11e47209a2112b4ecc8312b5cbddcf260d8577546f4a963fceb9334fca7ca56a"} Jul 10 13:15:07.628071 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:07.628011 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2r9v9" podStartSLOduration=1.627996289 podStartE2EDuration="1.627996289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:07.627253154 +0000 UTC m=+287.349179078" watchObservedRunningTime="2026-07-10 13:15:07.627996289 +0000 UTC m=+287.349922170" Jul 10 13:15:21.337788 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:21.337752 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:15:21.340914 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:21.340891 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 13:15:21.342174 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:21.342133 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:15:24.233597 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.233548 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2"] Jul 10 13:15:24.235918 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.235898 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.239894 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.239875 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 13:15:24.240385 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.240370 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 13:15:24.240459 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.240441 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 13:15:24.240515 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.240441 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 13:15:24.241144 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.241124 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tq2hx\"" Jul 10 13:15:24.260628 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.260604 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2"] Jul 10 13:15:24.285658 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.285630 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6201d837-2dc1-490b-b067-f5780e4d61eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.285790 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.285668 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6201d837-2dc1-490b-b067-f5780e4d61eb-manifests\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.285790 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.285688 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6201d837-2dc1-490b-b067-f5780e4d61eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.285790 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.285771 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmshf\" (UniqueName: \"kubernetes.io/projected/6201d837-2dc1-490b-b067-f5780e4d61eb-kube-api-access-wmshf\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.285940 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.285819 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6201d837-2dc1-490b-b067-f5780e4d61eb-tmp\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.386727 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.386689 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6201d837-2dc1-490b-b067-f5780e4d61eb-manifests\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.386727 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.386730 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6201d837-2dc1-490b-b067-f5780e4d61eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.386955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.386770 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmshf\" (UniqueName: \"kubernetes.io/projected/6201d837-2dc1-490b-b067-f5780e4d61eb-kube-api-access-wmshf\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.386955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.386806 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6201d837-2dc1-490b-b067-f5780e4d61eb-tmp\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.386955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.386896 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6201d837-2dc1-490b-b067-f5780e4d61eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.387178 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.387155 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6201d837-2dc1-490b-b067-f5780e4d61eb-manifests\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.389176 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.389145 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6201d837-2dc1-490b-b067-f5780e4d61eb-tmp\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.389302 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.389280 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6201d837-2dc1-490b-b067-f5780e4d61eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.389345 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.389291 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6201d837-2dc1-490b-b067-f5780e4d61eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.398098 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.398078 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmshf\" (UniqueName: \"kubernetes.io/projected/6201d837-2dc1-490b-b067-f5780e4d61eb-kube-api-access-wmshf\") pod \"opendatahub-operator-controller-manager-9ff96f8f8-w5xw2\" (UID: \"6201d837-2dc1-490b-b067-f5780e4d61eb\") " pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.545619 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.545580 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:24.746014 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.745987 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2"] Jul 10 13:15:24.746973 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:15:24.746950 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6201d837_2dc1_490b_b067_f5780e4d61eb.slice/crio-fe34aa762456d324477ed09b381b5e04edab7dadd528d8c57340c136b78795a1 WatchSource:0}: Error finding container fe34aa762456d324477ed09b381b5e04edab7dadd528d8c57340c136b78795a1: Status 404 returned error can't find the container with id fe34aa762456d324477ed09b381b5e04edab7dadd528d8c57340c136b78795a1 Jul 10 13:15:24.748797 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:24.748782 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 13:15:25.674698 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:25.674656 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" event={"ID":"6201d837-2dc1-490b-b067-f5780e4d61eb","Type":"ContainerStarted","Data":"fe34aa762456d324477ed09b381b5e04edab7dadd528d8c57340c136b78795a1"} Jul 10 13:15:28.688305 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:28.688266 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" event={"ID":"6201d837-2dc1-490b-b067-f5780e4d61eb","Type":"ContainerStarted","Data":"1bc953097f195bcdec6e8b0e6fb89844c607aa809b21e2a878cc465c5df855ff"} Jul 10 13:15:29.693351 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:29.693316 2571 generic.go:356] "Generic (PLEG): container finished" podID="6201d837-2dc1-490b-b067-f5780e4d61eb" containerID="1bc953097f195bcdec6e8b0e6fb89844c607aa809b21e2a878cc465c5df855ff" exitCode=0 Jul 10 13:15:29.693739 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:29.693412 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" event={"ID":"6201d837-2dc1-490b-b067-f5780e4d61eb","Type":"ContainerDied","Data":"1bc953097f195bcdec6e8b0e6fb89844c607aa809b21e2a878cc465c5df855ff"} Jul 10 13:15:30.699068 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:30.699020 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" event={"ID":"6201d837-2dc1-490b-b067-f5780e4d61eb","Type":"ContainerStarted","Data":"35f2f9b2f26348ca9d7405342cdf84324d0009394103cc1bca4832bef3186f39"} Jul 10 13:15:30.699453 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:30.699184 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:30.726171 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:30.726122 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" podStartSLOduration=2.374468798 podStartE2EDuration="6.726105499s" totalImagesPullingTime="3.289157611s" totalInitContainerRuntime="1.06247909s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:30.725095162 +0000 UTC m=+310.447021045" watchObservedRunningTime="2026-07-10 13:15:30.726105499 +0000 UTC m=+310.448031381" Jul 10 13:15:41.703751 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.703714 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk"] Jul 10 13:15:41.709089 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.709068 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9ff96f8f8-w5xw2" Jul 10 13:15:41.709207 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.709139 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.714155 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.714131 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 13:15:41.714155 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.714146 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 13:15:41.714299 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.714158 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 13:15:41.714299 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.714135 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2d9bg\"" Jul 10 13:15:41.714518 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.714505 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 13:15:41.714667 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.714650 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 13:15:41.746447 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.746416 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk"] Jul 10 13:15:41.836027 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.835994 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt5n2\" (UniqueName: \"kubernetes.io/projected/356252e9-2206-4ea6-a7fd-bc87d4f527b9-kube-api-access-kt5n2\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.836212 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.836039 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/356252e9-2206-4ea6-a7fd-bc87d4f527b9-cert\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.836212 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.836082 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/356252e9-2206-4ea6-a7fd-bc87d4f527b9-manager-config\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.836212 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.836179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/356252e9-2206-4ea6-a7fd-bc87d4f527b9-metrics-cert\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.937416 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.937387 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/356252e9-2206-4ea6-a7fd-bc87d4f527b9-cert\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.937598 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.937435 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/356252e9-2206-4ea6-a7fd-bc87d4f527b9-manager-config\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.937598 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.937490 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/356252e9-2206-4ea6-a7fd-bc87d4f527b9-metrics-cert\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.937598 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.937522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt5n2\" (UniqueName: \"kubernetes.io/projected/356252e9-2206-4ea6-a7fd-bc87d4f527b9-kube-api-access-kt5n2\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.938232 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.938211 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/356252e9-2206-4ea6-a7fd-bc87d4f527b9-manager-config\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.939848 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.939831 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/356252e9-2206-4ea6-a7fd-bc87d4f527b9-metrics-cert\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:41.939962 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:41.939946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/356252e9-2206-4ea6-a7fd-bc87d4f527b9-cert\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:42.025266 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:42.025239 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt5n2\" (UniqueName: \"kubernetes.io/projected/356252e9-2206-4ea6-a7fd-bc87d4f527b9-kube-api-access-kt5n2\") pod \"lws-controller-manager-7cffcb8bcf-9tktk\" (UID: \"356252e9-2206-4ea6-a7fd-bc87d4f527b9\") " pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:42.317779 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:42.317692 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:42.452484 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:42.452461 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk"] Jul 10 13:15:42.455022 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:15:42.454996 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356252e9_2206_4ea6_a7fd_bc87d4f527b9.slice/crio-f03b8869437f0a03360f28e7e8b70ebe05370513ff77817ff8657f14d51d78fc WatchSource:0}: Error finding container f03b8869437f0a03360f28e7e8b70ebe05370513ff77817ff8657f14d51d78fc: Status 404 returned error can't find the container with id f03b8869437f0a03360f28e7e8b70ebe05370513ff77817ff8657f14d51d78fc Jul 10 13:15:42.749164 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:42.749130 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" event={"ID":"356252e9-2206-4ea6-a7fd-bc87d4f527b9","Type":"ContainerStarted","Data":"f03b8869437f0a03360f28e7e8b70ebe05370513ff77817ff8657f14d51d78fc"} Jul 10 13:15:45.763730 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:45.763690 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" event={"ID":"356252e9-2206-4ea6-a7fd-bc87d4f527b9","Type":"ContainerStarted","Data":"c09c6ee49cbb96c58126d380441469b3d1fa41152d8049402801abf18260618c"} Jul 10 13:15:45.764152 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:45.763833 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:45.862146 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:45.862102 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" podStartSLOduration=2.553091751 podStartE2EDuration="4.862087648s" totalImagesPullingTime="2.308995897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:45.855379802 +0000 UTC m=+325.577305696" watchObservedRunningTime="2026-07-10 13:15:45.862087648 +0000 UTC m=+325.584013530" Jul 10 13:15:45.969667 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:45.969634 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z"] Jul 10 13:15:45.972169 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:45.972146 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:45.975292 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:45.975266 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 10 13:15:45.975449 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:45.975420 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-tls\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" Jul 10 13:15:45.975525 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:45.975458 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 10 13:15:45.975525 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:45.975500 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 10 13:15:45.975853 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:45.975828 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-dockercfg-vxbwm\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vxbwm\"" type="*v1.Secret" Jul 10 13:15:45.975892 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:45.975854 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-76f6cf8c47-rrg7z\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" podUID="c06b0362-393f-459c-9bba-9d5f2bc9ebc7" pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:45.993769 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:45.993746 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z"] Jul 10 13:15:46.073506 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.073423 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tls-certs\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.073506 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.073469 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tmp\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.073670 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.073561 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c79k\" (UniqueName: \"kubernetes.io/projected/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-kube-api-access-2c79k\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.174345 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.174306 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tmp\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.174515 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.174373 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c79k\" (UniqueName: \"kubernetes.io/projected/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-kube-api-access-2c79k\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.174515 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.174416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tls-certs\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.176562 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.176539 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tmp\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:46.883113 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:46.883080 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 13:15:47.093255 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.093218 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 13:15:47.134011 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.133938 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vxbwm\"" Jul 10 13:15:47.175274 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:47.175242 2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: failed to sync secret cache: timed out waiting for the condition Jul 10 13:15:47.175426 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:47.175346 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tls-certs podName:c06b0362-393f-459c-9bba-9d5f2bc9ebc7 nodeName:}" failed. No retries permitted until 2026-07-10 13:15:47.675324198 +0000 UTC m=+327.397250068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tls-certs") pod "kube-auth-proxy-76f6cf8c47-rrg7z" (UID: "c06b0362-393f-459c-9bba-9d5f2bc9ebc7") : failed to sync secret cache: timed out waiting for the condition Jul 10 13:15:47.204379 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:47.204350 2571 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 10 13:15:47.204511 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:47.204388 2571 projected.go:196] Error preparing data for projected volume kube-api-access-2c79k for pod openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z: failed to sync configmap cache: timed out waiting for the condition Jul 10 13:15:47.204511 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:15:47.204434 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-kube-api-access-2c79k podName:c06b0362-393f-459c-9bba-9d5f2bc9ebc7 nodeName:}" failed. No retries permitted until 2026-07-10 13:15:47.704419986 +0000 UTC m=+327.426345846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2c79k" (UniqueName: "kubernetes.io/projected/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-kube-api-access-2c79k") pod "kube-auth-proxy-76f6cf8c47-rrg7z" (UID: "c06b0362-393f-459c-9bba-9d5f2bc9ebc7") : failed to sync configmap cache: timed out waiting for the condition Jul 10 13:15:47.253269 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.253242 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 13:15:47.258450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.258430 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 13:15:47.688213 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.688172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tls-certs\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:47.690557 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.690529 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-tls-certs\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:47.789729 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.789689 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c79k\" (UniqueName: \"kubernetes.io/projected/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-kube-api-access-2c79k\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:47.792159 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:47.792140 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c79k\" (UniqueName: \"kubernetes.io/projected/c06b0362-393f-459c-9bba-9d5f2bc9ebc7-kube-api-access-2c79k\") pod \"kube-auth-proxy-76f6cf8c47-rrg7z\" (UID: \"c06b0362-393f-459c-9bba-9d5f2bc9ebc7\") " pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:48.082375 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:48.082340 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" Jul 10 13:15:48.243414 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:48.243379 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z"] Jul 10 13:15:48.244347 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:15:48.244325 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc06b0362_393f_459c_9bba_9d5f2bc9ebc7.slice/crio-cc3fb00a039e70a521cfef2d3355a0469374319eb8572ca9161b5958cea72528 WatchSource:0}: Error finding container cc3fb00a039e70a521cfef2d3355a0469374319eb8572ca9161b5958cea72528: Status 404 returned error can't find the container with id cc3fb00a039e70a521cfef2d3355a0469374319eb8572ca9161b5958cea72528 Jul 10 13:15:48.776972 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:48.776941 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" event={"ID":"c06b0362-393f-459c-9bba-9d5f2bc9ebc7","Type":"ContainerStarted","Data":"cc3fb00a039e70a521cfef2d3355a0469374319eb8572ca9161b5958cea72528"} Jul 10 13:15:51.789161 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:51.789126 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" event={"ID":"c06b0362-393f-459c-9bba-9d5f2bc9ebc7","Type":"ContainerStarted","Data":"cf6c5ec96b8cce8517b8b261970978a87586882f8e2e45c3deb0e97f55a7a49d"} Jul 10 13:15:51.833210 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:51.833165 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76f6cf8c47-rrg7z" podStartSLOduration=4.29306622 podStartE2EDuration="6.833151079s" totalImagesPullingTime="2.540084859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:51.831882745 +0000 UTC m=+331.553808627" watchObservedRunningTime="2026-07-10 13:15:51.833151079 +0000 UTC m=+331.555076974" Jul 10 13:15:56.770450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:56.770425 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cffcb8bcf-9tktk" Jul 10 13:15:58.459551 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.459521 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6"] Jul 10 13:15:58.540517 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.540485 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6"] Jul 10 13:15:58.540669 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.540593 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.572378 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.572356 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4xt52\"" Jul 10 13:15:58.576271 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.576253 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 13:15:58.576505 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.576486 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6pqr\" (UniqueName: \"kubernetes.io/projected/f0cb099f-958a-4f33-849d-ff079e918f13-kube-api-access-n6pqr\") pod \"llmisvc-controller-manager-777c6bd7d5-54zm6\" (UID: \"f0cb099f-958a-4f33-849d-ff079e918f13\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.576599 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.576586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0cb099f-958a-4f33-849d-ff079e918f13-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-54zm6\" (UID: \"f0cb099f-958a-4f33-849d-ff079e918f13\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.677121 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.677096 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0cb099f-958a-4f33-849d-ff079e918f13-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-54zm6\" (UID: \"f0cb099f-958a-4f33-849d-ff079e918f13\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.677222 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.677137 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6pqr\" (UniqueName: \"kubernetes.io/projected/f0cb099f-958a-4f33-849d-ff079e918f13-kube-api-access-n6pqr\") pod \"llmisvc-controller-manager-777c6bd7d5-54zm6\" (UID: \"f0cb099f-958a-4f33-849d-ff079e918f13\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.679544 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.679513 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0cb099f-958a-4f33-849d-ff079e918f13-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-54zm6\" (UID: \"f0cb099f-958a-4f33-849d-ff079e918f13\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.696296 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.696270 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6pqr\" (UniqueName: \"kubernetes.io/projected/f0cb099f-958a-4f33-849d-ff079e918f13-kube-api-access-n6pqr\") pod \"llmisvc-controller-manager-777c6bd7d5-54zm6\" (UID: \"f0cb099f-958a-4f33-849d-ff079e918f13\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.849820 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.849798 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:15:58.989585 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:58.989552 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6"] Jul 10 13:15:58.992284 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:15:58.992250 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0cb099f_958a_4f33_849d_ff079e918f13.slice/crio-a595406cae4baec6890449bbe5653110959dcb01b1b5bf5fbd76fc9593888e2e WatchSource:0}: Error finding container a595406cae4baec6890449bbe5653110959dcb01b1b5bf5fbd76fc9593888e2e: Status 404 returned error can't find the container with id a595406cae4baec6890449bbe5653110959dcb01b1b5bf5fbd76fc9593888e2e Jul 10 13:15:59.822929 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:15:59.822899 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" event={"ID":"f0cb099f-958a-4f33-849d-ff079e918f13","Type":"ContainerStarted","Data":"a595406cae4baec6890449bbe5653110959dcb01b1b5bf5fbd76fc9593888e2e"} Jul 10 13:16:03.842248 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:16:03.842207 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" event={"ID":"f0cb099f-958a-4f33-849d-ff079e918f13","Type":"ContainerStarted","Data":"5ad6d80728e9d717024578ac4d1adaf7a7a020600c5d73bf765fb9e06aa3cfc0"} Jul 10 13:16:03.842707 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:16:03.842360 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:16:03.913897 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:16:03.913845 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" podStartSLOduration=2.110509152 podStartE2EDuration="5.91382783s" totalImagesPullingTime="3.803318678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:16:03.912211422 +0000 UTC m=+343.634137304" watchObservedRunningTime="2026-07-10 13:16:03.91382783 +0000 UTC m=+343.635753712" Jul 10 13:16:34.847348 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:16:34.847268 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-54zm6" Jul 10 13:18:25.495191 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.495162 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:25.497450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.497428 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.502768 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.502746 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 13:18:25.503801 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.503745 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 13:18:25.503909 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.503745 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fd6r5\"" Jul 10 13:18:25.504413 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.504393 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 13:18:25.533198 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.533169 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:25.564308 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.564284 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d469e120-e39c-4e40-aa26-8475eadfc4ed-config-file\") pod \"limitador-limitador-64c94fd64f-8zh5n\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.564405 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.564346 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrw9f\" (UniqueName: \"kubernetes.io/projected/d469e120-e39c-4e40-aa26-8475eadfc4ed-kube-api-access-vrw9f\") pod \"limitador-limitador-64c94fd64f-8zh5n\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.577479 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.577454 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:25.664967 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.664946 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d469e120-e39c-4e40-aa26-8475eadfc4ed-config-file\") pod \"limitador-limitador-64c94fd64f-8zh5n\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.665075 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.665017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrw9f\" (UniqueName: \"kubernetes.io/projected/d469e120-e39c-4e40-aa26-8475eadfc4ed-kube-api-access-vrw9f\") pod \"limitador-limitador-64c94fd64f-8zh5n\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.665516 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.665498 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d469e120-e39c-4e40-aa26-8475eadfc4ed-config-file\") pod \"limitador-limitador-64c94fd64f-8zh5n\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.679418 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.679395 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrw9f\" (UniqueName: \"kubernetes.io/projected/d469e120-e39c-4e40-aa26-8475eadfc4ed-kube-api-access-vrw9f\") pod \"limitador-limitador-64c94fd64f-8zh5n\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.808801 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.808748 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:25.944335 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:25.944312 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:25.947012 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:18:25.946986 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd469e120_e39c_4e40_aa26_8475eadfc4ed.slice/crio-053cce562b34414ef201e5a162dd1e42f016959b76ebc8cae6988b95a074b27f WatchSource:0}: Error finding container 053cce562b34414ef201e5a162dd1e42f016959b76ebc8cae6988b95a074b27f: Status 404 returned error can't find the container with id 053cce562b34414ef201e5a162dd1e42f016959b76ebc8cae6988b95a074b27f Jul 10 13:18:26.423592 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.423557 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" event={"ID":"d469e120-e39c-4e40-aa26-8475eadfc4ed","Type":"ContainerStarted","Data":"053cce562b34414ef201e5a162dd1e42f016959b76ebc8cae6988b95a074b27f"} Jul 10 13:18:26.485483 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.485455 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4ggkc"] Jul 10 13:18:26.488505 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.488488 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:26.491236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.491214 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vb8dd\"" Jul 10 13:18:26.510401 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.510376 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ggkc"] Jul 10 13:18:26.571654 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.571630 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njzs9\" (UniqueName: \"kubernetes.io/projected/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02-kube-api-access-njzs9\") pod \"authorino-58cd955675-4ggkc\" (UID: \"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02\") " pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:26.672746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.672719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njzs9\" (UniqueName: \"kubernetes.io/projected/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02-kube-api-access-njzs9\") pod \"authorino-58cd955675-4ggkc\" (UID: \"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02\") " pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:26.693428 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.693364 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njzs9\" (UniqueName: \"kubernetes.io/projected/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02-kube-api-access-njzs9\") pod \"authorino-58cd955675-4ggkc\" (UID: \"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02\") " pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:26.797169 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.797135 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:26.835467 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.834635 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-f6cgd"] Jul 10 13:18:26.843065 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.843019 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:18:26.857555 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.857460 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f6cgd"] Jul 10 13:18:26.981497 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:26.981344 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57wfc\" (UniqueName: \"kubernetes.io/projected/20df590c-65eb-445e-8311-827b27f61eb4-kube-api-access-57wfc\") pod \"authorino-75d556bd77-f6cgd\" (UID: \"20df590c-65eb-445e-8311-827b27f61eb4\") " pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:18:27.004706 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.004642 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ggkc"] Jul 10 13:18:27.008464 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:18:27.008420 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0ded01c_2d7f_4b77_9a9e_d33d3a9a7a02.slice/crio-03f8c4e6ffcad3e307627209899ad9d962d3b37366ec44b16ad03a12007676ac WatchSource:0}: Error finding container 03f8c4e6ffcad3e307627209899ad9d962d3b37366ec44b16ad03a12007676ac: Status 404 returned error can't find the container with id 03f8c4e6ffcad3e307627209899ad9d962d3b37366ec44b16ad03a12007676ac Jul 10 13:18:27.082283 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.082247 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57wfc\" (UniqueName: \"kubernetes.io/projected/20df590c-65eb-445e-8311-827b27f61eb4-kube-api-access-57wfc\") pod \"authorino-75d556bd77-f6cgd\" (UID: \"20df590c-65eb-445e-8311-827b27f61eb4\") " pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:18:27.091553 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.091514 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57wfc\" (UniqueName: \"kubernetes.io/projected/20df590c-65eb-445e-8311-827b27f61eb4-kube-api-access-57wfc\") pod \"authorino-75d556bd77-f6cgd\" (UID: \"20df590c-65eb-445e-8311-827b27f61eb4\") " pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:18:27.157578 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.157545 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:18:27.312179 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.312139 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f6cgd"] Jul 10 13:18:27.315233 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:18:27.315209 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20df590c_65eb_445e_8311_827b27f61eb4.slice/crio-22f5ea65072257fbf8fd6941af398165a56e0485be4f331d39404f45b5d824e1 WatchSource:0}: Error finding container 22f5ea65072257fbf8fd6941af398165a56e0485be4f331d39404f45b5d824e1: Status 404 returned error can't find the container with id 22f5ea65072257fbf8fd6941af398165a56e0485be4f331d39404f45b5d824e1 Jul 10 13:18:27.430023 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.429986 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ggkc" event={"ID":"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02","Type":"ContainerStarted","Data":"03f8c4e6ffcad3e307627209899ad9d962d3b37366ec44b16ad03a12007676ac"} Jul 10 13:18:27.431113 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:27.431089 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f6cgd" event={"ID":"20df590c-65eb-445e-8311-827b27f61eb4","Type":"ContainerStarted","Data":"22f5ea65072257fbf8fd6941af398165a56e0485be4f331d39404f45b5d824e1"} Jul 10 13:18:29.445860 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:29.445771 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" event={"ID":"d469e120-e39c-4e40-aa26-8475eadfc4ed","Type":"ContainerStarted","Data":"2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda"} Jul 10 13:18:29.446607 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:29.446581 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:29.478750 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:29.478688 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" podStartSLOduration=1.375280814 podStartE2EDuration="4.47867231s" totalImagesPullingTime="3.103391496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:18:29.477941245 +0000 UTC m=+489.199867128" watchObservedRunningTime="2026-07-10 13:18:29.47867231 +0000 UTC m=+489.200598192" Jul 10 13:18:31.455869 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:31.455776 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f6cgd" event={"ID":"20df590c-65eb-445e-8311-827b27f61eb4","Type":"ContainerStarted","Data":"a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432"} Jul 10 13:18:31.457286 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:31.457260 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ggkc" event={"ID":"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02","Type":"ContainerStarted","Data":"68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba"} Jul 10 13:18:31.471236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:31.471142 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-f6cgd" podStartSLOduration=1.6506017640000001 podStartE2EDuration="5.471125812s" totalImagesPullingTime="3.820524048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:18:31.47103473 +0000 UTC m=+491.192960625" watchObservedRunningTime="2026-07-10 13:18:31.471125812 +0000 UTC m=+491.193051683" Jul 10 13:18:31.488018 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:31.487975 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4ggkc" podStartSLOduration=1.371311852 podStartE2EDuration="5.487961382s" totalImagesPullingTime="4.11664953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:18:31.487350369 +0000 UTC m=+491.209276258" watchObservedRunningTime="2026-07-10 13:18:31.487961382 +0000 UTC m=+491.209887263" Jul 10 13:18:31.521071 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:31.521020 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ggkc"] Jul 10 13:18:33.466195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:33.466156 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4ggkc" podUID="c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" containerName="authorino" containerID="cri-o://68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba" gracePeriod=30 Jul 10 13:18:33.711442 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:33.711419 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:33.841867 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:33.841816 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njzs9\" (UniqueName: \"kubernetes.io/projected/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02-kube-api-access-njzs9\") pod \"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02\" (UID: \"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02\") " Jul 10 13:18:33.843984 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:33.843962 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02-kube-api-access-njzs9" (OuterVolumeSpecName: "kube-api-access-njzs9") pod "c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" (UID: "c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02"). InnerVolumeSpecName "kube-api-access-njzs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:18:33.942478 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:33.942442 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njzs9\" (UniqueName: \"kubernetes.io/projected/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02-kube-api-access-njzs9\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:18:34.470763 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.470726 2571 generic.go:356] "Generic (PLEG): container finished" podID="c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" containerID="68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba" exitCode=0 Jul 10 13:18:34.471195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.470767 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ggkc" Jul 10 13:18:34.471195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.470807 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ggkc" event={"ID":"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02","Type":"ContainerDied","Data":"68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba"} Jul 10 13:18:34.471195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.470845 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ggkc" event={"ID":"c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02","Type":"ContainerDied","Data":"03f8c4e6ffcad3e307627209899ad9d962d3b37366ec44b16ad03a12007676ac"} Jul 10 13:18:34.471195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.470864 2571 scope.go:117] "RemoveContainer" containerID="68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba" Jul 10 13:18:34.478685 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.478670 2571 scope.go:117] "RemoveContainer" containerID="68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba" Jul 10 13:18:34.478925 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:18:34.478906 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba\": container with ID starting with 68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba not found: ID does not exist" containerID="68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba" Jul 10 13:18:34.478970 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.478933 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba"} err="failed to get container status \"68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba\": rpc error: code = NotFound desc = could not find container \"68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba\": container with ID starting with 68bac7fb2a75e09b1bc462366cda27e69d8d08cf5dc7b916e3248e28a3a129ba not found: ID does not exist" Jul 10 13:18:34.494667 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.494639 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ggkc"] Jul 10 13:18:34.507553 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:34.507532 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ggkc"] Jul 10 13:18:35.507768 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:35.507735 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" path="/var/lib/kubelet/pods/c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02/volumes" Jul 10 13:18:40.431910 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:40.431876 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:40.432366 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:40.432092 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" podUID="d469e120-e39c-4e40-aa26-8475eadfc4ed" containerName="limitador" containerID="cri-o://2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda" gracePeriod=30 Jul 10 13:18:40.432769 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:40.432670 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:40.980202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:40.980178 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:41.098976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.098951 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrw9f\" (UniqueName: \"kubernetes.io/projected/d469e120-e39c-4e40-aa26-8475eadfc4ed-kube-api-access-vrw9f\") pod \"d469e120-e39c-4e40-aa26-8475eadfc4ed\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " Jul 10 13:18:41.099135 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.099038 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d469e120-e39c-4e40-aa26-8475eadfc4ed-config-file\") pod \"d469e120-e39c-4e40-aa26-8475eadfc4ed\" (UID: \"d469e120-e39c-4e40-aa26-8475eadfc4ed\") " Jul 10 13:18:41.099324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.099300 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d469e120-e39c-4e40-aa26-8475eadfc4ed-config-file" (OuterVolumeSpecName: "config-file") pod "d469e120-e39c-4e40-aa26-8475eadfc4ed" (UID: "d469e120-e39c-4e40-aa26-8475eadfc4ed"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:18:41.100935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.100908 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d469e120-e39c-4e40-aa26-8475eadfc4ed-kube-api-access-vrw9f" (OuterVolumeSpecName: "kube-api-access-vrw9f") pod "d469e120-e39c-4e40-aa26-8475eadfc4ed" (UID: "d469e120-e39c-4e40-aa26-8475eadfc4ed"). InnerVolumeSpecName "kube-api-access-vrw9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:18:41.199881 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.199856 2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d469e120-e39c-4e40-aa26-8475eadfc4ed-config-file\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:18:41.199881 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.199877 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrw9f\" (UniqueName: \"kubernetes.io/projected/d469e120-e39c-4e40-aa26-8475eadfc4ed-kube-api-access-vrw9f\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:18:41.501576 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.501539 2571 generic.go:356] "Generic (PLEG): container finished" podID="d469e120-e39c-4e40-aa26-8475eadfc4ed" containerID="2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda" exitCode=0 Jul 10 13:18:41.502004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.501591 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" Jul 10 13:18:41.502004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.501644 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" event={"ID":"d469e120-e39c-4e40-aa26-8475eadfc4ed","Type":"ContainerDied","Data":"2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda"} Jul 10 13:18:41.502004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.501692 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8zh5n" event={"ID":"d469e120-e39c-4e40-aa26-8475eadfc4ed","Type":"ContainerDied","Data":"053cce562b34414ef201e5a162dd1e42f016959b76ebc8cae6988b95a074b27f"} Jul 10 13:18:41.502004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.501726 2571 scope.go:117] "RemoveContainer" containerID="2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda" Jul 10 13:18:41.512187 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.512167 2571 scope.go:117] "RemoveContainer" containerID="2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda" Jul 10 13:18:41.512805 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:18:41.512773 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda\": container with ID starting with 2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda not found: ID does not exist" containerID="2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda" Jul 10 13:18:41.512950 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.512917 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda"} err="failed to get container status \"2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda\": rpc error: code = NotFound desc = could not find container \"2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda\": container with ID starting with 2a208ebc5545d7d6066ff4640053eb299ef2c95ec8010004c7e6e47d60231bda not found: ID does not exist" Jul 10 13:18:41.529457 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.529436 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:41.534222 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:41.534201 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8zh5n"] Jul 10 13:18:43.508036 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:43.507996 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d469e120-e39c-4e40-aa26-8475eadfc4ed" path="/var/lib/kubelet/pods/d469e120-e39c-4e40-aa26-8475eadfc4ed/volumes" Jul 10 13:18:58.756703 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.756671 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fq8zf"] Jul 10 13:18:58.757096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.756968 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" containerName="authorino" Jul 10 13:18:58.757096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.756978 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" containerName="authorino" Jul 10 13:18:58.757096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.756987 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d469e120-e39c-4e40-aa26-8475eadfc4ed" containerName="limitador" Jul 10 13:18:58.757096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.756993 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d469e120-e39c-4e40-aa26-8475eadfc4ed" containerName="limitador" Jul 10 13:18:58.757096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.757041 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d469e120-e39c-4e40-aa26-8475eadfc4ed" containerName="limitador" Jul 10 13:18:58.757096 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.757049 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ded01c-2d7f-4b77-9a9e-d33d3a9a7a02" containerName="authorino" Jul 10 13:18:58.760066 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.760032 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:18:58.775149 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.775125 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fq8zf"] Jul 10 13:18:58.829161 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.829137 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vldxm\" (UniqueName: \"kubernetes.io/projected/249743d9-42ee-4f01-9387-2a0283353564-kube-api-access-vldxm\") pod \"authorino-5d4b67c446-fq8zf\" (UID: \"249743d9-42ee-4f01-9387-2a0283353564\") " pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:18:58.929653 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.929627 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vldxm\" (UniqueName: \"kubernetes.io/projected/249743d9-42ee-4f01-9387-2a0283353564-kube-api-access-vldxm\") pod \"authorino-5d4b67c446-fq8zf\" (UID: \"249743d9-42ee-4f01-9387-2a0283353564\") " pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:18:58.940470 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:58.940447 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vldxm\" (UniqueName: \"kubernetes.io/projected/249743d9-42ee-4f01-9387-2a0283353564-kube-api-access-vldxm\") pod \"authorino-5d4b67c446-fq8zf\" (UID: \"249743d9-42ee-4f01-9387-2a0283353564\") " pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:18:59.046641 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.046565 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fq8zf"] Jul 10 13:18:59.046788 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.046774 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:18:59.083492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.083457 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56545f7fd5-pbbhb"] Jul 10 13:18:59.088296 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.088274 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.091336 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.091318 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 13:18:59.101184 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.101161 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56545f7fd5-pbbhb"] Jul 10 13:18:59.131538 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.131502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-tls-cert\") pod \"authorino-56545f7fd5-pbbhb\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.131673 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.131550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj7hd\" (UniqueName: \"kubernetes.io/projected/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-kube-api-access-vj7hd\") pod \"authorino-56545f7fd5-pbbhb\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.179981 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.179903 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fq8zf"] Jul 10 13:18:59.182560 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:18:59.182534 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod249743d9_42ee_4f01_9387_2a0283353564.slice/crio-c71d047c687b23a4e379ed27879e00c4b795c47ec9e600b655674167518320e9 WatchSource:0}: Error finding container c71d047c687b23a4e379ed27879e00c4b795c47ec9e600b655674167518320e9: Status 404 returned error can't find the container with id c71d047c687b23a4e379ed27879e00c4b795c47ec9e600b655674167518320e9 Jul 10 13:18:59.232414 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.232372 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-tls-cert\") pod \"authorino-56545f7fd5-pbbhb\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.232552 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.232428 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj7hd\" (UniqueName: \"kubernetes.io/projected/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-kube-api-access-vj7hd\") pod \"authorino-56545f7fd5-pbbhb\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.234719 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.234701 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-tls-cert\") pod \"authorino-56545f7fd5-pbbhb\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.242390 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.242372 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj7hd\" (UniqueName: \"kubernetes.io/projected/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-kube-api-access-vj7hd\") pod \"authorino-56545f7fd5-pbbhb\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.399870 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.399786 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:18:59.532400 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.532332 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56545f7fd5-pbbhb"] Jul 10 13:18:59.534386 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:18:59.534358 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfed3e141_a467_43d8_bdb6_cdf1babb3ba8.slice/crio-db0d8c952cb2993adcf74262e4246b47f2334e2922f64e878ba696d2f2ba3ddc WatchSource:0}: Error finding container db0d8c952cb2993adcf74262e4246b47f2334e2922f64e878ba696d2f2ba3ddc: Status 404 returned error can't find the container with id db0d8c952cb2993adcf74262e4246b47f2334e2922f64e878ba696d2f2ba3ddc Jul 10 13:18:59.574716 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.574686 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" event={"ID":"249743d9-42ee-4f01-9387-2a0283353564","Type":"ContainerStarted","Data":"c71d047c687b23a4e379ed27879e00c4b795c47ec9e600b655674167518320e9"} Jul 10 13:18:59.575841 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:18:59.575816 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" event={"ID":"fed3e141-a467-43d8-bdb6-cdf1babb3ba8","Type":"ContainerStarted","Data":"db0d8c952cb2993adcf74262e4246b47f2334e2922f64e878ba696d2f2ba3ddc"} Jul 10 13:19:00.581962 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.581909 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" event={"ID":"fed3e141-a467-43d8-bdb6-cdf1babb3ba8","Type":"ContainerStarted","Data":"c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13"} Jul 10 13:19:00.583290 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.583267 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" event={"ID":"249743d9-42ee-4f01-9387-2a0283353564","Type":"ContainerStarted","Data":"bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07"} Jul 10 13:19:00.583378 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.583316 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" podUID="249743d9-42ee-4f01-9387-2a0283353564" containerName="authorino" containerID="cri-o://bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07" gracePeriod=30 Jul 10 13:19:00.599224 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.599186 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" podStartSLOduration=1.22291294 podStartE2EDuration="1.599172773s" totalImagesPullingTime="376.259833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:00.597842189 +0000 UTC m=+520.319768071" watchObservedRunningTime="2026-07-10 13:19:00.599172773 +0000 UTC m=+520.321098647" Jul 10 13:19:00.630912 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.630865 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" podStartSLOduration=2.254830909 podStartE2EDuration="2.630846909s" totalImagesPullingTime="376.016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:00.629299492 +0000 UTC m=+520.351225374" watchObservedRunningTime="2026-07-10 13:19:00.630846909 +0000 UTC m=+520.352772795" Jul 10 13:19:00.637007 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.636982 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f6cgd"] Jul 10 13:19:00.637165 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.637139 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-f6cgd" podUID="20df590c-65eb-445e-8311-827b27f61eb4" containerName="authorino" containerID="cri-o://a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432" gracePeriod=30 Jul 10 13:19:00.911617 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.911594 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:19:00.914268 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:00.914246 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:19:01.049512 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.049479 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vldxm\" (UniqueName: \"kubernetes.io/projected/249743d9-42ee-4f01-9387-2a0283353564-kube-api-access-vldxm\") pod \"249743d9-42ee-4f01-9387-2a0283353564\" (UID: \"249743d9-42ee-4f01-9387-2a0283353564\") " Jul 10 13:19:01.049512 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.049515 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57wfc\" (UniqueName: \"kubernetes.io/projected/20df590c-65eb-445e-8311-827b27f61eb4-kube-api-access-57wfc\") pod \"20df590c-65eb-445e-8311-827b27f61eb4\" (UID: \"20df590c-65eb-445e-8311-827b27f61eb4\") " Jul 10 13:19:01.051535 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.051509 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20df590c-65eb-445e-8311-827b27f61eb4-kube-api-access-57wfc" (OuterVolumeSpecName: "kube-api-access-57wfc") pod "20df590c-65eb-445e-8311-827b27f61eb4" (UID: "20df590c-65eb-445e-8311-827b27f61eb4"). InnerVolumeSpecName "kube-api-access-57wfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:19:01.051650 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.051568 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/249743d9-42ee-4f01-9387-2a0283353564-kube-api-access-vldxm" (OuterVolumeSpecName: "kube-api-access-vldxm") pod "249743d9-42ee-4f01-9387-2a0283353564" (UID: "249743d9-42ee-4f01-9387-2a0283353564"). InnerVolumeSpecName "kube-api-access-vldxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:19:01.150663 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.150594 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vldxm\" (UniqueName: \"kubernetes.io/projected/249743d9-42ee-4f01-9387-2a0283353564-kube-api-access-vldxm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:19:01.150663 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.150618 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57wfc\" (UniqueName: \"kubernetes.io/projected/20df590c-65eb-445e-8311-827b27f61eb4-kube-api-access-57wfc\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:19:01.588499 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.588459 2571 generic.go:356] "Generic (PLEG): container finished" podID="249743d9-42ee-4f01-9387-2a0283353564" containerID="bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07" exitCode=0 Jul 10 13:19:01.588935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.588510 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" Jul 10 13:19:01.588935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.588515 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" event={"ID":"249743d9-42ee-4f01-9387-2a0283353564","Type":"ContainerDied","Data":"bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07"} Jul 10 13:19:01.588935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.588562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fq8zf" event={"ID":"249743d9-42ee-4f01-9387-2a0283353564","Type":"ContainerDied","Data":"c71d047c687b23a4e379ed27879e00c4b795c47ec9e600b655674167518320e9"} Jul 10 13:19:01.588935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.588584 2571 scope.go:117] "RemoveContainer" containerID="bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07" Jul 10 13:19:01.590028 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.589887 2571 generic.go:356] "Generic (PLEG): container finished" podID="20df590c-65eb-445e-8311-827b27f61eb4" containerID="a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432" exitCode=0 Jul 10 13:19:01.590028 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.589941 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f6cgd" Jul 10 13:19:01.590028 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.589945 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f6cgd" event={"ID":"20df590c-65eb-445e-8311-827b27f61eb4","Type":"ContainerDied","Data":"a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432"} Jul 10 13:19:01.590028 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.589988 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f6cgd" event={"ID":"20df590c-65eb-445e-8311-827b27f61eb4","Type":"ContainerDied","Data":"22f5ea65072257fbf8fd6941af398165a56e0485be4f331d39404f45b5d824e1"} Jul 10 13:19:01.597497 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.597475 2571 scope.go:117] "RemoveContainer" containerID="bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07" Jul 10 13:19:01.597784 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:19:01.597762 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07\": container with ID starting with bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07 not found: ID does not exist" containerID="bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07" Jul 10 13:19:01.597882 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.597795 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07"} err="failed to get container status \"bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07\": rpc error: code = NotFound desc = could not find container \"bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07\": container with ID starting with bbe18a4d550c69c04507663393e2c8361ceab37242d03bca00b835cb598bbb07 not found: ID does not exist" Jul 10 13:19:01.597882 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.597818 2571 scope.go:117] "RemoveContainer" containerID="a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432" Jul 10 13:19:01.607438 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.607418 2571 scope.go:117] "RemoveContainer" containerID="a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432" Jul 10 13:19:01.608010 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:19:01.607988 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432\": container with ID starting with a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432 not found: ID does not exist" containerID="a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432" Jul 10 13:19:01.608121 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.608019 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432"} err="failed to get container status \"a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432\": rpc error: code = NotFound desc = could not find container \"a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432\": container with ID starting with a26fc7f9a6f1dc23d0414b1f9ac48b4c59c62a6c13c0e0a4db8575b605aaa432 not found: ID does not exist" Jul 10 13:19:01.609110 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.609088 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fq8zf"] Jul 10 13:19:01.611935 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.611915 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fq8zf"] Jul 10 13:19:01.624230 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.624212 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f6cgd"] Jul 10 13:19:01.634466 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:01.634448 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f6cgd"] Jul 10 13:19:03.507452 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:03.507422 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20df590c-65eb-445e-8311-827b27f61eb4" path="/var/lib/kubelet/pods/20df590c-65eb-445e-8311-827b27f61eb4/volumes" Jul 10 13:19:03.507829 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:03.507710 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="249743d9-42ee-4f01-9387-2a0283353564" path="/var/lib/kubelet/pods/249743d9-42ee-4f01-9387-2a0283353564/volumes" Jul 10 13:19:05.266374 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266336 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58fffff548-h6szq"] Jul 10 13:19:05.266922 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266782 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20df590c-65eb-445e-8311-827b27f61eb4" containerName="authorino" Jul 10 13:19:05.266922 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266801 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="20df590c-65eb-445e-8311-827b27f61eb4" containerName="authorino" Jul 10 13:19:05.266922 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266820 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="249743d9-42ee-4f01-9387-2a0283353564" containerName="authorino" Jul 10 13:19:05.266922 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266829 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="249743d9-42ee-4f01-9387-2a0283353564" containerName="authorino" Jul 10 13:19:05.266922 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266924 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="249743d9-42ee-4f01-9387-2a0283353564" containerName="authorino" Jul 10 13:19:05.267192 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.266937 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="20df590c-65eb-445e-8311-827b27f61eb4" containerName="authorino" Jul 10 13:19:05.273884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.273858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.278021 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.277996 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 13:19:05.278147 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.278113 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 13:19:05.278461 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.278444 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mclg9\"" Jul 10 13:19:05.284242 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.278970 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4gw2\" (UniqueName: \"kubernetes.io/projected/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-kube-api-access-b4gw2\") pod \"maas-controller-58fffff548-h6szq\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.284242 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.279135 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-webhook-cert\") pod \"maas-controller-58fffff548-h6szq\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.289315 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.289292 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58fffff548-h6szq"] Jul 10 13:19:05.379812 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.379778 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4gw2\" (UniqueName: \"kubernetes.io/projected/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-kube-api-access-b4gw2\") pod \"maas-controller-58fffff548-h6szq\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.379957 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.379865 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-webhook-cert\") pod \"maas-controller-58fffff548-h6szq\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.382415 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.382383 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-webhook-cert\") pod \"maas-controller-58fffff548-h6szq\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.391549 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.391521 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4gw2\" (UniqueName: \"kubernetes.io/projected/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-kube-api-access-b4gw2\") pod \"maas-controller-58fffff548-h6szq\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.582210 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.582141 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-668b649c84-h5hz4"] Jul 10 13:19:05.584435 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.584420 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.589713 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.589692 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:05.617091 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.617050 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-668b649c84-h5hz4"] Jul 10 13:19:05.682907 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.682875 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6df7486-ac7f-4f87-918d-267baa277719-webhook-cert\") pod \"maas-controller-668b649c84-h5hz4\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.683046 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.682931 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zw5x\" (UniqueName: \"kubernetes.io/projected/a6df7486-ac7f-4f87-918d-267baa277719-kube-api-access-6zw5x\") pod \"maas-controller-668b649c84-h5hz4\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.725367 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.725339 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58fffff548-h6szq"] Jul 10 13:19:05.730401 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:19:05.730374 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2342380_dfa4_4fe3_a568_d0a0d22b2c82.slice/crio-9ecc8c86759405b7f29e28d9160dc7df5bf093fa3ec973052336aa41ef232e6c WatchSource:0}: Error finding container 9ecc8c86759405b7f29e28d9160dc7df5bf093fa3ec973052336aa41ef232e6c: Status 404 returned error can't find the container with id 9ecc8c86759405b7f29e28d9160dc7df5bf093fa3ec973052336aa41ef232e6c Jul 10 13:19:05.783884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.783856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6df7486-ac7f-4f87-918d-267baa277719-webhook-cert\") pod \"maas-controller-668b649c84-h5hz4\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.783979 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.783908 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zw5x\" (UniqueName: \"kubernetes.io/projected/a6df7486-ac7f-4f87-918d-267baa277719-kube-api-access-6zw5x\") pod \"maas-controller-668b649c84-h5hz4\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.786199 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.786182 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6df7486-ac7f-4f87-918d-267baa277719-webhook-cert\") pod \"maas-controller-668b649c84-h5hz4\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.796868 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.796847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zw5x\" (UniqueName: \"kubernetes.io/projected/a6df7486-ac7f-4f87-918d-267baa277719-kube-api-access-6zw5x\") pod \"maas-controller-668b649c84-h5hz4\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:05.894575 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:05.894525 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:06.037037 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:06.037014 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-668b649c84-h5hz4"] Jul 10 13:19:06.038550 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:19:06.038520 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6df7486_ac7f_4f87_918d_267baa277719.slice/crio-154324d2ab696242a746efa235d1c33f0429c484a44be2cfb3237b7174728754 WatchSource:0}: Error finding container 154324d2ab696242a746efa235d1c33f0429c484a44be2cfb3237b7174728754: Status 404 returned error can't find the container with id 154324d2ab696242a746efa235d1c33f0429c484a44be2cfb3237b7174728754 Jul 10 13:19:06.617084 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:06.617033 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-668b649c84-h5hz4" event={"ID":"a6df7486-ac7f-4f87-918d-267baa277719","Type":"ContainerStarted","Data":"154324d2ab696242a746efa235d1c33f0429c484a44be2cfb3237b7174728754"} Jul 10 13:19:06.618452 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:06.618422 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fffff548-h6szq" event={"ID":"d2342380-dfa4-4fe3-a568-d0a0d22b2c82","Type":"ContainerStarted","Data":"9ecc8c86759405b7f29e28d9160dc7df5bf093fa3ec973052336aa41ef232e6c"} Jul 10 13:19:08.630151 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:08.630115 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-668b649c84-h5hz4" event={"ID":"a6df7486-ac7f-4f87-918d-267baa277719","Type":"ContainerStarted","Data":"9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f"} Jul 10 13:19:08.630518 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:08.630182 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:08.631533 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:08.631512 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fffff548-h6szq" event={"ID":"d2342380-dfa4-4fe3-a568-d0a0d22b2c82","Type":"ContainerStarted","Data":"5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f"} Jul 10 13:19:08.631639 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:08.631627 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:08.654901 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:08.654852 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-668b649c84-h5hz4" podStartSLOduration=1.5375210080000001 podStartE2EDuration="3.654836461s" totalImagesPullingTime="2.117315453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:08.654442196 +0000 UTC m=+528.376368079" watchObservedRunningTime="2026-07-10 13:19:08.654836461 +0000 UTC m=+528.376762343" Jul 10 13:19:08.683031 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:08.682995 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58fffff548-h6szq" podStartSLOduration=1.26653188 podStartE2EDuration="3.682982015s" totalImagesPullingTime="2.416450135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:08.682098351 +0000 UTC m=+528.404024233" watchObservedRunningTime="2026-07-10 13:19:08.682982015 +0000 UTC m=+528.404907896" Jul 10 13:19:19.641407 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.641330 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:19:19.641789 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.641540 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:19.719477 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.719447 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58fffff548-h6szq"] Jul 10 13:19:19.719680 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.719646 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58fffff548-h6szq" podUID="d2342380-dfa4-4fe3-a568-d0a0d22b2c82" containerName="manager" containerID="cri-o://5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f" gracePeriod=30 Jul 10 13:19:19.959457 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.959429 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:19.995294 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.995265 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4gw2\" (UniqueName: \"kubernetes.io/projected/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-kube-api-access-b4gw2\") pod \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " Jul 10 13:19:19.995457 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.995323 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-webhook-cert\") pod \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\" (UID: \"d2342380-dfa4-4fe3-a568-d0a0d22b2c82\") " Jul 10 13:19:19.997458 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.997425 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d2342380-dfa4-4fe3-a568-d0a0d22b2c82" (UID: "d2342380-dfa4-4fe3-a568-d0a0d22b2c82"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:19:19.997562 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:19.997467 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-kube-api-access-b4gw2" (OuterVolumeSpecName: "kube-api-access-b4gw2") pod "d2342380-dfa4-4fe3-a568-d0a0d22b2c82" (UID: "d2342380-dfa4-4fe3-a568-d0a0d22b2c82"). InnerVolumeSpecName "kube-api-access-b4gw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:19:20.096575 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.096539 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4gw2\" (UniqueName: \"kubernetes.io/projected/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-kube-api-access-b4gw2\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:19:20.096575 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.096568 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2342380-dfa4-4fe3-a568-d0a0d22b2c82-webhook-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:19:20.682736 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.682705 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2342380-dfa4-4fe3-a568-d0a0d22b2c82" containerID="5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f" exitCode=0 Jul 10 13:19:20.683109 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.682743 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fffff548-h6szq" Jul 10 13:19:20.683109 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.682785 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fffff548-h6szq" event={"ID":"d2342380-dfa4-4fe3-a568-d0a0d22b2c82","Type":"ContainerDied","Data":"5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f"} Jul 10 13:19:20.683109 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.682827 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fffff548-h6szq" event={"ID":"d2342380-dfa4-4fe3-a568-d0a0d22b2c82","Type":"ContainerDied","Data":"9ecc8c86759405b7f29e28d9160dc7df5bf093fa3ec973052336aa41ef232e6c"} Jul 10 13:19:20.683109 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.682849 2571 scope.go:117] "RemoveContainer" containerID="5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f" Jul 10 13:19:20.692199 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.692103 2571 scope.go:117] "RemoveContainer" containerID="5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f" Jul 10 13:19:20.692526 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:19:20.692495 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f\": container with ID starting with 5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f not found: ID does not exist" containerID="5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f" Jul 10 13:19:20.692632 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.692533 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f"} err="failed to get container status \"5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f\": rpc error: code = NotFound desc = could not find container \"5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f\": container with ID starting with 5e562b7d19c47a4d74acae803ed1a82f98b4c61662d43c6d7e408c4fec3a041f not found: ID does not exist" Jul 10 13:19:20.713634 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.713612 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58fffff548-h6szq"] Jul 10 13:19:20.726143 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:20.726122 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58fffff548-h6szq"] Jul 10 13:19:21.507278 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:21.507246 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2342380-dfa4-4fe3-a568-d0a0d22b2c82" path="/var/lib/kubelet/pods/d2342380-dfa4-4fe3-a568-d0a0d22b2c82/volumes" Jul 10 13:19:41.439159 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.439123 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b74766796-gj6d7"] Jul 10 13:19:41.439630 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.439448 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2342380-dfa4-4fe3-a568-d0a0d22b2c82" containerName="manager" Jul 10 13:19:41.439630 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.439459 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2342380-dfa4-4fe3-a568-d0a0d22b2c82" containerName="manager" Jul 10 13:19:41.439630 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.439511 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2342380-dfa4-4fe3-a568-d0a0d22b2c82" containerName="manager" Jul 10 13:19:41.445079 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.445040 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.449362 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.449342 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 13:19:41.449484 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.449387 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 13:19:41.449484 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.449425 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 13:19:41.450029 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.450013 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pzz4r\"" Jul 10 13:19:41.466654 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.466632 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b74766796-gj6d7"] Jul 10 13:19:41.569997 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.569968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe04d0e-0360-4e86-9236-5cc468e624b3-maas-api-tls\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.570176 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.570039 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwl64\" (UniqueName: \"kubernetes.io/projected/afe04d0e-0360-4e86-9236-5cc468e624b3-kube-api-access-lwl64\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.570176 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.570169 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe04d0e-0360-4e86-9236-5cc468e624b3-openshift-service-ca\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.671562 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.671438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe04d0e-0360-4e86-9236-5cc468e624b3-maas-api-tls\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.671562 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.671518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwl64\" (UniqueName: \"kubernetes.io/projected/afe04d0e-0360-4e86-9236-5cc468e624b3-kube-api-access-lwl64\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.671562 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.671565 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe04d0e-0360-4e86-9236-5cc468e624b3-openshift-service-ca\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.671850 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:19:41.671644 2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 10 13:19:41.671850 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:19:41.671722 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe04d0e-0360-4e86-9236-5cc468e624b3-maas-api-tls podName:afe04d0e-0360-4e86-9236-5cc468e624b3 nodeName:}" failed. No retries permitted until 2026-07-10 13:19:42.171702828 +0000 UTC m=+561.893628710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/afe04d0e-0360-4e86-9236-5cc468e624b3-maas-api-tls") pod "maas-api-b74766796-gj6d7" (UID: "afe04d0e-0360-4e86-9236-5cc468e624b3") : secret "maas-api-serving-cert" not found Jul 10 13:19:41.672306 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.672272 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe04d0e-0360-4e86-9236-5cc468e624b3-openshift-service-ca\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:41.685820 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:41.685798 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwl64\" (UniqueName: \"kubernetes.io/projected/afe04d0e-0360-4e86-9236-5cc468e624b3-kube-api-access-lwl64\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:42.176874 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.176846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe04d0e-0360-4e86-9236-5cc468e624b3-maas-api-tls\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:42.179031 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.179011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe04d0e-0360-4e86-9236-5cc468e624b3-maas-api-tls\") pod \"maas-api-b74766796-gj6d7\" (UID: \"afe04d0e-0360-4e86-9236-5cc468e624b3\") " pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:42.186405 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.186379 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nvw74"] Jul 10 13:19:42.190379 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.190364 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.193316 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.193296 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wls6h\"" Jul 10 13:19:42.193583 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.193564 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 13:19:42.199229 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.199208 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nvw74"] Jul 10 13:19:42.237073 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.237032 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-54f59"] Jul 10 13:19:42.240797 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.240781 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.252360 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.252339 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-54f59"] Jul 10 13:19:42.278337 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.278310 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dfvx\" (UniqueName: \"kubernetes.io/projected/2fd2f80e-6219-4b37-8d41-824bc12a9637-kube-api-access-6dfvx\") pod \"payload-processing-58f86c7798-nvw74\" (UID: \"2fd2f80e-6219-4b37-8d41-824bc12a9637\") " pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.278475 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.278342 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fd2f80e-6219-4b37-8d41-824bc12a9637-plugins-config-volume\") pod \"payload-processing-58f86c7798-nvw74\" (UID: \"2fd2f80e-6219-4b37-8d41-824bc12a9637\") " pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.355220 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.355192 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:42.379091 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.379044 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cab35f80-a85b-4b19-935c-ef4450e3f8a8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-54f59\" (UID: \"cab35f80-a85b-4b19-935c-ef4450e3f8a8\") " pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.379233 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.379176 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjcsb\" (UniqueName: \"kubernetes.io/projected/cab35f80-a85b-4b19-935c-ef4450e3f8a8-kube-api-access-xjcsb\") pod \"payload-pre-processing-7964457696-54f59\" (UID: \"cab35f80-a85b-4b19-935c-ef4450e3f8a8\") " pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.379297 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.379232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dfvx\" (UniqueName: \"kubernetes.io/projected/2fd2f80e-6219-4b37-8d41-824bc12a9637-kube-api-access-6dfvx\") pod \"payload-processing-58f86c7798-nvw74\" (UID: \"2fd2f80e-6219-4b37-8d41-824bc12a9637\") " pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.379297 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.379278 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fd2f80e-6219-4b37-8d41-824bc12a9637-plugins-config-volume\") pod \"payload-processing-58f86c7798-nvw74\" (UID: \"2fd2f80e-6219-4b37-8d41-824bc12a9637\") " pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.379918 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.379899 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fd2f80e-6219-4b37-8d41-824bc12a9637-plugins-config-volume\") pod \"payload-processing-58f86c7798-nvw74\" (UID: \"2fd2f80e-6219-4b37-8d41-824bc12a9637\") " pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.387742 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.387713 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dfvx\" (UniqueName: \"kubernetes.io/projected/2fd2f80e-6219-4b37-8d41-824bc12a9637-kube-api-access-6dfvx\") pod \"payload-processing-58f86c7798-nvw74\" (UID: \"2fd2f80e-6219-4b37-8d41-824bc12a9637\") " pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.480336 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.480305 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cab35f80-a85b-4b19-935c-ef4450e3f8a8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-54f59\" (UID: \"cab35f80-a85b-4b19-935c-ef4450e3f8a8\") " pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.480680 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.480350 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjcsb\" (UniqueName: \"kubernetes.io/projected/cab35f80-a85b-4b19-935c-ef4450e3f8a8-kube-api-access-xjcsb\") pod \"payload-pre-processing-7964457696-54f59\" (UID: \"cab35f80-a85b-4b19-935c-ef4450e3f8a8\") " pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.480956 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.480938 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cab35f80-a85b-4b19-935c-ef4450e3f8a8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-54f59\" (UID: \"cab35f80-a85b-4b19-935c-ef4450e3f8a8\") " pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.488491 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.488466 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjcsb\" (UniqueName: \"kubernetes.io/projected/cab35f80-a85b-4b19-935c-ef4450e3f8a8-kube-api-access-xjcsb\") pod \"payload-pre-processing-7964457696-54f59\" (UID: \"cab35f80-a85b-4b19-935c-ef4450e3f8a8\") " pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.492072 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.492033 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b74766796-gj6d7"] Jul 10 13:19:42.494563 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:19:42.494537 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe04d0e_0360_4e86_9236_5cc468e624b3.slice/crio-2f4455a200951cf30456e22330eda46bb39a4013ad22bd097289169e76b84388 WatchSource:0}: Error finding container 2f4455a200951cf30456e22330eda46bb39a4013ad22bd097289169e76b84388: Status 404 returned error can't find the container with id 2f4455a200951cf30456e22330eda46bb39a4013ad22bd097289169e76b84388 Jul 10 13:19:42.501588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.501569 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:42.550200 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.550168 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:42.646177 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.646152 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nvw74"] Jul 10 13:19:42.648549 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:19:42.648502 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd2f80e_6219_4b37_8d41_824bc12a9637.slice/crio-2490726ec76f638e4f9fa712caff00e23a426efba1b603c96f1f13348865a1d1 WatchSource:0}: Error finding container 2490726ec76f638e4f9fa712caff00e23a426efba1b603c96f1f13348865a1d1: Status 404 returned error can't find the container with id 2490726ec76f638e4f9fa712caff00e23a426efba1b603c96f1f13348865a1d1 Jul 10 13:19:42.701454 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.701278 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-54f59"] Jul 10 13:19:42.704217 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:19:42.704183 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab35f80_a85b_4b19_935c_ef4450e3f8a8.slice/crio-83d6f8a2eedbb18dd79c42d793d260aeaa569dd680ed3e14e5e3019fee24a369 WatchSource:0}: Error finding container 83d6f8a2eedbb18dd79c42d793d260aeaa569dd680ed3e14e5e3019fee24a369: Status 404 returned error can't find the container with id 83d6f8a2eedbb18dd79c42d793d260aeaa569dd680ed3e14e5e3019fee24a369 Jul 10 13:19:42.777343 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.777311 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" event={"ID":"afe04d0e-0360-4e86-9236-5cc468e624b3","Type":"ContainerStarted","Data":"2f4455a200951cf30456e22330eda46bb39a4013ad22bd097289169e76b84388"} Jul 10 13:19:42.778541 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.778501 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" event={"ID":"cab35f80-a85b-4b19-935c-ef4450e3f8a8","Type":"ContainerStarted","Data":"83d6f8a2eedbb18dd79c42d793d260aeaa569dd680ed3e14e5e3019fee24a369"} Jul 10 13:19:42.779587 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:42.779559 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" event={"ID":"2fd2f80e-6219-4b37-8d41-824bc12a9637","Type":"ContainerStarted","Data":"2490726ec76f638e4f9fa712caff00e23a426efba1b603c96f1f13348865a1d1"} Jul 10 13:19:44.795386 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.795298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" event={"ID":"afe04d0e-0360-4e86-9236-5cc468e624b3","Type":"ContainerStarted","Data":"8d3fd4becc201c0d2b0809dad89a8a1c27a745910abc427ef68ecb293c9ed173"} Jul 10 13:19:44.795827 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.795398 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:44.797086 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.797038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" event={"ID":"cab35f80-a85b-4b19-935c-ef4450e3f8a8","Type":"ContainerStarted","Data":"77f1645a1ecb204753b2696a9885e3bf9e747bc81dee5f2828d0f22a91b709fa"} Jul 10 13:19:44.797221 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.797110 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:19:44.798530 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.798501 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" event={"ID":"2fd2f80e-6219-4b37-8d41-824bc12a9637","Type":"ContainerStarted","Data":"083784da7ae9b2bac5e002d6b1048d35dde7a5415967711771c0570e681a4b83"} Jul 10 13:19:44.798634 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.798554 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:44.814429 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.814390 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" podStartSLOduration=1.7555802360000001 podStartE2EDuration="3.814377425s" totalImagesPullingTime="2.058797189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:44.813416868 +0000 UTC m=+564.535342753" watchObservedRunningTime="2026-07-10 13:19:44.814377425 +0000 UTC m=+564.536303307" Jul 10 13:19:44.856860 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.856822 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" podStartSLOduration=0.950467067 podStartE2EDuration="2.856809381s" totalImagesPullingTime="1.906342314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:19:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:44.856023215 +0000 UTC m=+564.577949098" watchObservedRunningTime="2026-07-10 13:19:44.856809381 +0000 UTC m=+564.578735263" Jul 10 13:19:44.856967 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:44.856901 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" podStartSLOduration=1.008494599 podStartE2EDuration="2.856895381s" totalImagesPullingTime="1.848400782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:19:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:19:44.836293269 +0000 UTC m=+564.558219153" watchObservedRunningTime="2026-07-10 13:19:44.856895381 +0000 UTC m=+564.578821266" Jul 10 13:19:50.808257 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:50.808222 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b74766796-gj6d7" Jul 10 13:19:55.804735 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:55.804700 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-nvw74" Jul 10 13:19:55.805266 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:19:55.804799 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-54f59" Jul 10 13:20:15.519813 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.519779 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t"] Jul 10 13:20:15.523193 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.523176 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.527164 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.527141 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4jbww\"" Jul 10 13:20:15.527281 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.527141 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 13:20:15.527420 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.527407 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 13:20:15.535431 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.535414 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 13:20:15.540786 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.540764 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.540886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.540792 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83b80762-8628-426c-9144-e560c8930e71-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.540886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.540820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2fmr\" (UniqueName: \"kubernetes.io/projected/83b80762-8628-426c-9144-e560c8930e71-kube-api-access-b2fmr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.540886 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.540841 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.541014 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.540903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.541014 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.540949 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.548518 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.548496 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t"] Jul 10 13:20:15.642192 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642163 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2fmr\" (UniqueName: \"kubernetes.io/projected/83b80762-8628-426c-9144-e560c8930e71-kube-api-access-b2fmr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642336 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642211 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642336 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642250 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642336 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642285 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642463 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642347 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642463 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642370 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83b80762-8628-426c-9144-e560c8930e71-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642687 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642670 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642767 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642752 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.642824 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.642785 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.644646 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.644627 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83b80762-8628-426c-9144-e560c8930e71-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.644827 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.644812 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83b80762-8628-426c-9144-e560c8930e71-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.658085 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.658043 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2fmr\" (UniqueName: \"kubernetes.io/projected/83b80762-8628-426c-9144-e560c8930e71-kube-api-access-b2fmr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wqr2t\" (UID: \"83b80762-8628-426c-9144-e560c8930e71\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.833719 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.833650 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:15.971159 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:15.971027 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t"] Jul 10 13:20:15.974550 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:20:15.974517 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b80762_8628_426c_9144_e560c8930e71.slice/crio-49e789cf6e4a7b58f5a7f6a877070061fd0f25f17b52f35c4781898eb45d2d3b WatchSource:0}: Error finding container 49e789cf6e4a7b58f5a7f6a877070061fd0f25f17b52f35c4781898eb45d2d3b: Status 404 returned error can't find the container with id 49e789cf6e4a7b58f5a7f6a877070061fd0f25f17b52f35c4781898eb45d2d3b Jul 10 13:20:16.952820 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:16.952781 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" event={"ID":"83b80762-8628-426c-9144-e560c8930e71","Type":"ContainerStarted","Data":"49e789cf6e4a7b58f5a7f6a877070061fd0f25f17b52f35c4781898eb45d2d3b"} Jul 10 13:20:17.958907 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:17.958874 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" event={"ID":"83b80762-8628-426c-9144-e560c8930e71","Type":"ContainerStarted","Data":"8c459c845ab99f8a201ab14a437c1411a94b7b75063251b75c053e00607082d9"} Jul 10 13:20:18.018422 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:18.018370 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" podStartSLOduration=1.916140338 podStartE2EDuration="3.018353416s" totalImagesPullingTime="1.102213078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:20:18.016179201 +0000 UTC m=+597.738105083" watchObservedRunningTime="2026-07-10 13:20:18.018353416 +0000 UTC m=+597.740279297" Jul 10 13:20:21.366963 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:21.366937 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:20:21.367494 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:21.367471 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:20:25.834189 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:25.834142 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:25.836551 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:25.834211 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:25.846690 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:25.846668 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:26.014539 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:26.014509 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wqr2t" Jul 10 13:20:34.471894 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.471859 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg"] Jul 10 13:20:34.475403 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.475386 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.479348 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.479326 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 13:20:34.506554 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.506530 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg"] Jul 10 13:20:34.609828 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.609794 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.609983 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.609833 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56sv6\" (UniqueName: \"kubernetes.io/projected/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-kube-api-access-56sv6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.609983 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.609897 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.609983 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.609962 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.610122 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.610012 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.610122 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.610046 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.710972 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.710932 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.710972 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.710972 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.710995 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.711019 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56sv6\" (UniqueName: \"kubernetes.io/projected/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-kube-api-access-56sv6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.711090 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711218 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.711153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711464 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.711437 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711509 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.711469 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.711554 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.711538 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.713326 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.713304 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.713489 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.713471 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.723037 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.722982 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56sv6\" (UniqueName: \"kubernetes.io/projected/9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0-kube-api-access-56sv6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg\" (UID: \"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.785463 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.785429 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:34.932616 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:20:34.932588 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca4b5cb_7f0a_4f8e_87be_af78ad55e8c0.slice/crio-d301f6d0fc2f12f296929d18d96344e25ada6bca160dd8bf29c5e70cd4df9350 WatchSource:0}: Error finding container d301f6d0fc2f12f296929d18d96344e25ada6bca160dd8bf29c5e70cd4df9350: Status 404 returned error can't find the container with id d301f6d0fc2f12f296929d18d96344e25ada6bca160dd8bf29c5e70cd4df9350 Jul 10 13:20:34.933996 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.933974 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 13:20:34.939814 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:34.939791 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg"] Jul 10 13:20:35.043870 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:35.043831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" event={"ID":"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0","Type":"ContainerStarted","Data":"d301f6d0fc2f12f296929d18d96344e25ada6bca160dd8bf29c5e70cd4df9350"} Jul 10 13:20:36.050064 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:36.050024 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" event={"ID":"9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0","Type":"ContainerStarted","Data":"8b47898fb035a8d73ad352f57ee5fc6f3d2c14496b7902dd3ab4f73d86c067a5"} Jul 10 13:20:36.085551 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:36.085496 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" podStartSLOduration=1.888309761 podStartE2EDuration="2.085481474s" totalImagesPullingTime="197.171713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:20:36.084247867 +0000 UTC m=+615.806173749" watchObservedRunningTime="2026-07-10 13:20:36.085481474 +0000 UTC m=+615.807407385" Jul 10 13:20:44.785766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:44.785680 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:44.785766 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:44.785744 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:44.798420 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:44.798393 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:20:45.101186 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:20:45.101110 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg" Jul 10 13:21:04.710605 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.710572 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js"] Jul 10 13:21:04.716476 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.716410 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.720430 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.720405 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 13:21:04.753713 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.753682 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js"] Jul 10 13:21:04.775094 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.775042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.775246 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.775111 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.775246 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.775170 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftwvz\" (UniqueName: \"kubernetes.io/projected/7407534a-a843-4aa9-918e-06693d4b5f62-kube-api-access-ftwvz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.775323 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.775246 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7407534a-a843-4aa9-918e-06693d4b5f62-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.775323 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.775287 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.775323 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.775304 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876108 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876050 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftwvz\" (UniqueName: \"kubernetes.io/projected/7407534a-a843-4aa9-918e-06693d4b5f62-kube-api-access-ftwvz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876303 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876143 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7407534a-a843-4aa9-918e-06693d4b5f62-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876303 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876303 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876192 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876303 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876277 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876505 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.876853 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.876826 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.877111 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.877087 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.877228 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.877205 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.878858 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.878834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7407534a-a843-4aa9-918e-06693d4b5f62-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.878995 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.878976 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7407534a-a843-4aa9-918e-06693d4b5f62-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:04.890631 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:04.890608 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftwvz\" (UniqueName: \"kubernetes.io/projected/7407534a-a843-4aa9-918e-06693d4b5f62-kube-api-access-ftwvz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg7js\" (UID: \"7407534a-a843-4aa9-918e-06693d4b5f62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:05.026560 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:05.026519 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:05.173625 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:05.173588 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js"] Jul 10 13:21:05.178836 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:21:05.178807 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7407534a_a843_4aa9_918e_06693d4b5f62.slice/crio-306702896be87b3636f0d5a24db10fb2d8131e248487e399b9f8703c837e6714 WatchSource:0}: Error finding container 306702896be87b3636f0d5a24db10fb2d8131e248487e399b9f8703c837e6714: Status 404 returned error can't find the container with id 306702896be87b3636f0d5a24db10fb2d8131e248487e399b9f8703c837e6714 Jul 10 13:21:06.189732 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:06.189689 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" event={"ID":"7407534a-a843-4aa9-918e-06693d4b5f62","Type":"ContainerStarted","Data":"d8e8148ce01468e2056bc4f3ab4668c03e03251ad95f2519ca59772a7d4eaa17"} Jul 10 13:21:06.190210 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:06.189785 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" event={"ID":"7407534a-a843-4aa9-918e-06693d4b5f62","Type":"ContainerStarted","Data":"306702896be87b3636f0d5a24db10fb2d8131e248487e399b9f8703c837e6714"} Jul 10 13:21:06.217559 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:06.217512 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" podStartSLOduration=1.980352414 podStartE2EDuration="2.217496425s" totalImagesPullingTime="237.144011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:21:06.215210281 +0000 UTC m=+645.937136163" watchObservedRunningTime="2026-07-10 13:21:06.217496425 +0000 UTC m=+645.939422307" Jul 10 13:21:08.636902 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.636869 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn"] Jul 10 13:21:08.640382 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.640365 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.648273 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.648253 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 13:21:08.695785 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.695760 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn"] Jul 10 13:21:08.712945 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.712919 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.713097 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.712963 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28b3cedd-db76-4446-8243-85699e42a39d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.713097 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.712993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.713193 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.713093 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brn68\" (UniqueName: \"kubernetes.io/projected/28b3cedd-db76-4446-8243-85699e42a39d-kube-api-access-brn68\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.713193 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.713126 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.713299 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.713223 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.814695 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.814660 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.814845 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.814702 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brn68\" (UniqueName: \"kubernetes.io/projected/28b3cedd-db76-4446-8243-85699e42a39d-kube-api-access-brn68\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.814845 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.814794 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.814924 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.814853 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.814924 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.814908 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.815017 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.814958 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28b3cedd-db76-4446-8243-85699e42a39d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.815222 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.815201 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.815356 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.815220 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.815356 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.815332 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.817264 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.817242 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28b3cedd-db76-4446-8243-85699e42a39d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.817406 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.817390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28b3cedd-db76-4446-8243-85699e42a39d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.834718 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.834691 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brn68\" (UniqueName: \"kubernetes.io/projected/28b3cedd-db76-4446-8243-85699e42a39d-kube-api-access-brn68\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn\" (UID: \"28b3cedd-db76-4446-8243-85699e42a39d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:08.949078 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:08.948985 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:09.083889 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:09.083866 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn"] Jul 10 13:21:09.086259 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:21:09.086230 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28b3cedd_db76_4446_8243_85699e42a39d.slice/crio-0fd6dcace2e9a0f25d570c62549a7c232071f6b185cc44b87fb98bec430ff934 WatchSource:0}: Error finding container 0fd6dcace2e9a0f25d570c62549a7c232071f6b185cc44b87fb98bec430ff934: Status 404 returned error can't find the container with id 0fd6dcace2e9a0f25d570c62549a7c232071f6b185cc44b87fb98bec430ff934 Jul 10 13:21:09.205324 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:09.205242 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" event={"ID":"28b3cedd-db76-4446-8243-85699e42a39d","Type":"ContainerStarted","Data":"0fd6dcace2e9a0f25d570c62549a7c232071f6b185cc44b87fb98bec430ff934"} Jul 10 13:21:10.211103 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:10.211048 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" event={"ID":"28b3cedd-db76-4446-8243-85699e42a39d","Type":"ContainerStarted","Data":"93d20b7d6e5a8163d8c81170bde9e72da00dd7f06c09af24d918114236f49a98"} Jul 10 13:21:10.229573 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:10.229518 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" podStartSLOduration=2.012222492 podStartE2EDuration="2.229503517s" totalImagesPullingTime="217.281025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:21:10.228089744 +0000 UTC m=+649.950015627" watchObservedRunningTime="2026-07-10 13:21:10.229503517 +0000 UTC m=+649.951429451" Jul 10 13:21:15.027129 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:15.027092 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:15.027528 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:15.027152 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:15.039256 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:15.039233 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:15.244739 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:15.244712 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg7js" Jul 10 13:21:18.950074 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:18.950030 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:18.950622 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:18.950101 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:18.962342 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:18.962318 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:19.261945 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:19.261921 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn" Jul 10 13:21:55.421487 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.421454 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f5b9bf496-5wggg"] Jul 10 13:21:55.424706 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.424687 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.433080 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.433034 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f5b9bf496-5wggg"] Jul 10 13:21:55.521085 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.521038 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/510fbbbc-ceb9-4c54-a3fa-35485a2129fe-tls-cert\") pod \"authorino-f5b9bf496-5wggg\" (UID: \"510fbbbc-ceb9-4c54-a3fa-35485a2129fe\") " pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.521188 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.521110 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7bvj\" (UniqueName: \"kubernetes.io/projected/510fbbbc-ceb9-4c54-a3fa-35485a2129fe-kube-api-access-p7bvj\") pod \"authorino-f5b9bf496-5wggg\" (UID: \"510fbbbc-ceb9-4c54-a3fa-35485a2129fe\") " pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.621995 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.621961 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/510fbbbc-ceb9-4c54-a3fa-35485a2129fe-tls-cert\") pod \"authorino-f5b9bf496-5wggg\" (UID: \"510fbbbc-ceb9-4c54-a3fa-35485a2129fe\") " pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.622117 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.622030 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7bvj\" (UniqueName: \"kubernetes.io/projected/510fbbbc-ceb9-4c54-a3fa-35485a2129fe-kube-api-access-p7bvj\") pod \"authorino-f5b9bf496-5wggg\" (UID: \"510fbbbc-ceb9-4c54-a3fa-35485a2129fe\") " pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.624323 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.624299 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/510fbbbc-ceb9-4c54-a3fa-35485a2129fe-tls-cert\") pod \"authorino-f5b9bf496-5wggg\" (UID: \"510fbbbc-ceb9-4c54-a3fa-35485a2129fe\") " pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.629870 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.629847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7bvj\" (UniqueName: \"kubernetes.io/projected/510fbbbc-ceb9-4c54-a3fa-35485a2129fe-kube-api-access-p7bvj\") pod \"authorino-f5b9bf496-5wggg\" (UID: \"510fbbbc-ceb9-4c54-a3fa-35485a2129fe\") " pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.733946 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.733854 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f5b9bf496-5wggg" Jul 10 13:21:55.873776 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:55.873741 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f5b9bf496-5wggg"] Jul 10 13:21:55.878894 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:21:55.878868 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod510fbbbc_ceb9_4c54_a3fa_35485a2129fe.slice/crio-a81aec6e09f63a212f6e041948173478a739ec21680846301db1edb8bbf958c5 WatchSource:0}: Error finding container a81aec6e09f63a212f6e041948173478a739ec21680846301db1edb8bbf958c5: Status 404 returned error can't find the container with id a81aec6e09f63a212f6e041948173478a739ec21680846301db1edb8bbf958c5 Jul 10 13:21:56.416169 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:56.416136 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f5b9bf496-5wggg" event={"ID":"510fbbbc-ceb9-4c54-a3fa-35485a2129fe","Type":"ContainerStarted","Data":"a81aec6e09f63a212f6e041948173478a739ec21680846301db1edb8bbf958c5"} Jul 10 13:21:57.425599 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.425557 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f5b9bf496-5wggg" event={"ID":"510fbbbc-ceb9-4c54-a3fa-35485a2129fe","Type":"ContainerStarted","Data":"589125e17f883862fa1cddc98e309064006bae55eb37861576cca43411b6a63d"} Jul 10 13:21:57.442649 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.442602 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f5b9bf496-5wggg" podStartSLOduration=1.880737342 podStartE2EDuration="2.442586526s" totalImagesPullingTime="561.849184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:21:57.442556444 +0000 UTC m=+697.164482337" watchObservedRunningTime="2026-07-10 13:21:57.442586526 +0000 UTC m=+697.164512407" Jul 10 13:21:57.476905 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.476871 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56545f7fd5-pbbhb"] Jul 10 13:21:57.477116 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.477080 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" podUID="fed3e141-a467-43d8-bdb6-cdf1babb3ba8" containerName="authorino" containerID="cri-o://c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13" gracePeriod=30 Jul 10 13:21:57.731697 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.731668 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:21:57.848078 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.846301 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-tls-cert\") pod \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " Jul 10 13:21:57.848078 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.846442 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj7hd\" (UniqueName: \"kubernetes.io/projected/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-kube-api-access-vj7hd\") pod \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\" (UID: \"fed3e141-a467-43d8-bdb6-cdf1babb3ba8\") " Jul 10 13:21:57.856243 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.856183 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-kube-api-access-vj7hd" (OuterVolumeSpecName: "kube-api-access-vj7hd") pod "fed3e141-a467-43d8-bdb6-cdf1babb3ba8" (UID: "fed3e141-a467-43d8-bdb6-cdf1babb3ba8"). InnerVolumeSpecName "kube-api-access-vj7hd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:21:57.872728 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.872692 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "fed3e141-a467-43d8-bdb6-cdf1babb3ba8" (UID: "fed3e141-a467-43d8-bdb6-cdf1babb3ba8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:21:57.954074 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.949039 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj7hd\" (UniqueName: \"kubernetes.io/projected/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-kube-api-access-vj7hd\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:21:57.954074 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:57.949094 2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fed3e141-a467-43d8-bdb6-cdf1babb3ba8-tls-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:21:58.431472 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.431431 2571 generic.go:356] "Generic (PLEG): container finished" podID="fed3e141-a467-43d8-bdb6-cdf1babb3ba8" containerID="c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13" exitCode=0 Jul 10 13:21:58.431949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.431529 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" event={"ID":"fed3e141-a467-43d8-bdb6-cdf1babb3ba8","Type":"ContainerDied","Data":"c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13"} Jul 10 13:21:58.431949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.431589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" event={"ID":"fed3e141-a467-43d8-bdb6-cdf1babb3ba8","Type":"ContainerDied","Data":"db0d8c952cb2993adcf74262e4246b47f2334e2922f64e878ba696d2f2ba3ddc"} Jul 10 13:21:58.431949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.431617 2571 scope.go:117] "RemoveContainer" containerID="c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13" Jul 10 13:21:58.431949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.431537 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56545f7fd5-pbbhb" Jul 10 13:21:58.440174 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.440161 2571 scope.go:117] "RemoveContainer" containerID="c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13" Jul 10 13:21:58.440447 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:21:58.440429 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13\": container with ID starting with c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13 not found: ID does not exist" containerID="c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13" Jul 10 13:21:58.440496 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.440456 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13"} err="failed to get container status \"c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13\": rpc error: code = NotFound desc = could not find container \"c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13\": container with ID starting with c65a54650c4fc93dd5bd8355e2ad91f7351ddca6890e09029628cadf1d99db13 not found: ID does not exist" Jul 10 13:21:58.462011 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.461987 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56545f7fd5-pbbhb"] Jul 10 13:21:58.468978 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:58.468950 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56545f7fd5-pbbhb"] Jul 10 13:21:59.507545 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:21:59.507512 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed3e141-a467-43d8-bdb6-cdf1babb3ba8" path="/var/lib/kubelet/pods/fed3e141-a467-43d8-bdb6-cdf1babb3ba8/volumes" Jul 10 13:23:13.328220 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.328185 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-668b649c84-h5hz4"] Jul 10 13:23:13.328713 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.328366 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-668b649c84-h5hz4" podUID="a6df7486-ac7f-4f87-918d-267baa277719" containerName="manager" containerID="cri-o://9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f" gracePeriod=30 Jul 10 13:23:13.580299 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.580242 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:23:13.606904 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.606882 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6df7486-ac7f-4f87-918d-267baa277719-webhook-cert\") pod \"a6df7486-ac7f-4f87-918d-267baa277719\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " Jul 10 13:23:13.607039 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.606952 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zw5x\" (UniqueName: \"kubernetes.io/projected/a6df7486-ac7f-4f87-918d-267baa277719-kube-api-access-6zw5x\") pod \"a6df7486-ac7f-4f87-918d-267baa277719\" (UID: \"a6df7486-ac7f-4f87-918d-267baa277719\") " Jul 10 13:23:13.608892 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.608867 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6df7486-ac7f-4f87-918d-267baa277719-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a6df7486-ac7f-4f87-918d-267baa277719" (UID: "a6df7486-ac7f-4f87-918d-267baa277719"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:23:13.609009 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.608989 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6df7486-ac7f-4f87-918d-267baa277719-kube-api-access-6zw5x" (OuterVolumeSpecName: "kube-api-access-6zw5x") pod "a6df7486-ac7f-4f87-918d-267baa277719" (UID: "a6df7486-ac7f-4f87-918d-267baa277719"). InnerVolumeSpecName "kube-api-access-6zw5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:23:13.708224 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.708200 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6df7486-ac7f-4f87-918d-267baa277719-webhook-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:23:13.708224 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.708223 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zw5x\" (UniqueName: \"kubernetes.io/projected/a6df7486-ac7f-4f87-918d-267baa277719-kube-api-access-6zw5x\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:23:13.770410 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.770380 2571 generic.go:356] "Generic (PLEG): container finished" podID="a6df7486-ac7f-4f87-918d-267baa277719" containerID="9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f" exitCode=0 Jul 10 13:23:13.770536 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.770425 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-668b649c84-h5hz4" Jul 10 13:23:13.770536 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.770457 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-668b649c84-h5hz4" event={"ID":"a6df7486-ac7f-4f87-918d-267baa277719","Type":"ContainerDied","Data":"9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f"} Jul 10 13:23:13.770536 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.770506 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-668b649c84-h5hz4" event={"ID":"a6df7486-ac7f-4f87-918d-267baa277719","Type":"ContainerDied","Data":"154324d2ab696242a746efa235d1c33f0429c484a44be2cfb3237b7174728754"} Jul 10 13:23:13.770536 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.770531 2571 scope.go:117] "RemoveContainer" containerID="9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f" Jul 10 13:23:13.779377 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.779361 2571 scope.go:117] "RemoveContainer" containerID="9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f" Jul 10 13:23:13.779628 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:23:13.779610 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f\": container with ID starting with 9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f not found: ID does not exist" containerID="9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f" Jul 10 13:23:13.779689 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.779640 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f"} err="failed to get container status \"9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f\": rpc error: code = NotFound desc = could not find container \"9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f\": container with ID starting with 9ac9fd6b23705a01cc9d5c2cf61580be2e2fccd20add9276889ed0f5d60ac20f not found: ID does not exist" Jul 10 13:23:13.801357 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.801320 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-668b649c84-h5hz4"] Jul 10 13:23:13.804443 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:13.804418 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-668b649c84-h5hz4"] Jul 10 13:23:14.601608 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.601571 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-668b649c84-jkhpt"] Jul 10 13:23:14.602140 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.602120 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed3e141-a467-43d8-bdb6-cdf1babb3ba8" containerName="authorino" Jul 10 13:23:14.602140 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.602141 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed3e141-a467-43d8-bdb6-cdf1babb3ba8" containerName="authorino" Jul 10 13:23:14.602281 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.602168 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6df7486-ac7f-4f87-918d-267baa277719" containerName="manager" Jul 10 13:23:14.602281 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.602177 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6df7486-ac7f-4f87-918d-267baa277719" containerName="manager" Jul 10 13:23:14.602376 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.602295 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6df7486-ac7f-4f87-918d-267baa277719" containerName="manager" Jul 10 13:23:14.602376 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.602313 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed3e141-a467-43d8-bdb6-cdf1babb3ba8" containerName="authorino" Jul 10 13:23:14.606834 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.606813 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.610203 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.610185 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 13:23:14.611192 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.611176 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 13:23:14.611282 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.611216 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mclg9\"" Jul 10 13:23:14.614921 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.614900 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-668b649c84-jkhpt"] Jul 10 13:23:14.615219 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.615203 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b93ef924-abcd-4a3c-a5e9-a8e228484c1b-webhook-cert\") pod \"maas-controller-668b649c84-jkhpt\" (UID: \"b93ef924-abcd-4a3c-a5e9-a8e228484c1b\") " pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.615297 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.615252 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qds55\" (UniqueName: \"kubernetes.io/projected/b93ef924-abcd-4a3c-a5e9-a8e228484c1b-kube-api-access-qds55\") pod \"maas-controller-668b649c84-jkhpt\" (UID: \"b93ef924-abcd-4a3c-a5e9-a8e228484c1b\") " pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.715605 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.715575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b93ef924-abcd-4a3c-a5e9-a8e228484c1b-webhook-cert\") pod \"maas-controller-668b649c84-jkhpt\" (UID: \"b93ef924-abcd-4a3c-a5e9-a8e228484c1b\") " pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.715725 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.715646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qds55\" (UniqueName: \"kubernetes.io/projected/b93ef924-abcd-4a3c-a5e9-a8e228484c1b-kube-api-access-qds55\") pod \"maas-controller-668b649c84-jkhpt\" (UID: \"b93ef924-abcd-4a3c-a5e9-a8e228484c1b\") " pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.717908 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.717883 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b93ef924-abcd-4a3c-a5e9-a8e228484c1b-webhook-cert\") pod \"maas-controller-668b649c84-jkhpt\" (UID: \"b93ef924-abcd-4a3c-a5e9-a8e228484c1b\") " pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.723953 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.723931 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qds55\" (UniqueName: \"kubernetes.io/projected/b93ef924-abcd-4a3c-a5e9-a8e228484c1b-kube-api-access-qds55\") pod \"maas-controller-668b649c84-jkhpt\" (UID: \"b93ef924-abcd-4a3c-a5e9-a8e228484c1b\") " pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:14.918179 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:14.918098 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:15.065494 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:15.065466 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-668b649c84-jkhpt"] Jul 10 13:23:15.068162 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:23:15.068127 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93ef924_abcd_4a3c_a5e9_a8e228484c1b.slice/crio-c3b4d184f66a58a35860e1007311d2951d28701998a9d483c47cb4ef96e4c842 WatchSource:0}: Error finding container c3b4d184f66a58a35860e1007311d2951d28701998a9d483c47cb4ef96e4c842: Status 404 returned error can't find the container with id c3b4d184f66a58a35860e1007311d2951d28701998a9d483c47cb4ef96e4c842 Jul 10 13:23:15.508117 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:15.508078 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6df7486-ac7f-4f87-918d-267baa277719" path="/var/lib/kubelet/pods/a6df7486-ac7f-4f87-918d-267baa277719/volumes" Jul 10 13:23:15.782247 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:15.782219 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-668b649c84-jkhpt" event={"ID":"b93ef924-abcd-4a3c-a5e9-a8e228484c1b","Type":"ContainerStarted","Data":"37bda5d5b8736cd4704bbceb01f95fa0ceb8d03e24e243b0e93c9b917ab60325"} Jul 10 13:23:15.782572 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:15.782270 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-668b649c84-jkhpt" event={"ID":"b93ef924-abcd-4a3c-a5e9-a8e228484c1b","Type":"ContainerStarted","Data":"c3b4d184f66a58a35860e1007311d2951d28701998a9d483c47cb4ef96e4c842"} Jul 10 13:23:15.782572 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:15.782339 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:26.793543 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:26.793512 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-668b649c84-jkhpt" Jul 10 13:23:26.822193 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:23:26.822148 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-668b649c84-jkhpt" podStartSLOduration=12.200003703 podStartE2EDuration="12.822134492s" totalImagesPullingTime="622.130789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:23:15.811878464 +0000 UTC m=+775.533804357" watchObservedRunningTime="2026-07-10 13:23:26.822134492 +0000 UTC m=+786.544060373" Jul 10 13:25:21.400558 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:25:21.400526 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:25:21.401592 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:25:21.401572 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:30:21.430507 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:30:21.430476 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:30:21.432985 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:30:21.432964 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:35:21.459175 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:35:21.459143 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:35:21.462825 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:35:21.462807 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:40:21.487029 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:40:21.486999 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:40:21.491491 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:40:21.491470 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:41:00.027959 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.027922 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b"] Jul 10 13:41:00.031527 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.031504 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.034353 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.034328 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-faacd6-serving-cert\"" Jul 10 13:41:00.043522 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.043499 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b"] Jul 10 13:41:00.093891 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.093864 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1f9c99-91b1-4720-91a2-252fad40a755-openshift-service-ca\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.094011 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.093911 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc1f9c99-91b1-4720-91a2-252fad40a755-maas-api-tls\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.094011 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.093938 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vb8g\" (UniqueName: \"kubernetes.io/projected/cc1f9c99-91b1-4720-91a2-252fad40a755-kube-api-access-2vb8g\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.194943 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.194916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1f9c99-91b1-4720-91a2-252fad40a755-openshift-service-ca\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.195047 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.194963 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc1f9c99-91b1-4720-91a2-252fad40a755-maas-api-tls\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.195128 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.195104 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vb8g\" (UniqueName: \"kubernetes.io/projected/cc1f9c99-91b1-4720-91a2-252fad40a755-kube-api-access-2vb8g\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.196036 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.196002 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1f9c99-91b1-4720-91a2-252fad40a755-openshift-service-ca\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.202653 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.202629 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc1f9c99-91b1-4720-91a2-252fad40a755-maas-api-tls\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.205126 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.205102 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vb8g\" (UniqueName: \"kubernetes.io/projected/cc1f9c99-91b1-4720-91a2-252fad40a755-kube-api-access-2vb8g\") pod \"maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.343460 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.343391 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:00.471438 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.471381 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b"] Jul 10 13:41:00.473774 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:41:00.473750 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1f9c99_91b1_4720_91a2_252fad40a755.slice/crio-49560afdae984429a9c81aac55144fdc8e83b7167ce707aa8b982b168128b9e3 WatchSource:0}: Error finding container 49560afdae984429a9c81aac55144fdc8e83b7167ce707aa8b982b168128b9e3: Status 404 returned error can't find the container with id 49560afdae984429a9c81aac55144fdc8e83b7167ce707aa8b982b168128b9e3 Jul 10 13:41:00.475242 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.475222 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 13:41:00.620199 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:00.620125 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" event={"ID":"cc1f9c99-91b1-4720-91a2-252fad40a755","Type":"ContainerStarted","Data":"49560afdae984429a9c81aac55144fdc8e83b7167ce707aa8b982b168128b9e3"} Jul 10 13:41:01.625867 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:01.625829 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" event={"ID":"cc1f9c99-91b1-4720-91a2-252fad40a755","Type":"ContainerStarted","Data":"776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56"} Jul 10 13:41:01.626363 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:01.625967 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:01.645202 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:01.645158 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" podStartSLOduration=1.380698615 podStartE2EDuration="1.645140711s" totalImagesPullingTime="264.442096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:41:01.643174303 +0000 UTC m=+1841.365100196" watchObservedRunningTime="2026-07-10 13:41:01.645140711 +0000 UTC m=+1841.367066594" Jul 10 13:41:07.635240 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:07.635208 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:41:43.338416 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.338356 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv"] Jul 10 13:41:43.341924 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.341904 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.345422 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.345395 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-faacd6\"/\"kube-root-ca.crt\"" Jul 10 13:41:43.345422 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.345416 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-faacd6\"/\"default-dockercfg-fgmfc\"" Jul 10 13:41:43.345602 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.345442 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-faacd6\"/\"test-model-faacd6-kserve-self-signed-certs\"" Jul 10 13:41:43.345602 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.345531 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-faacd6\"/\"openshift-service-ca.crt\"" Jul 10 13:41:43.351273 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.351251 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv"] Jul 10 13:41:43.363323 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.363300 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-dshm\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.363439 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.363348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-tmp-dir\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.363439 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.363380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57lzf\" (UniqueName: \"kubernetes.io/projected/45c9942b-32f5-491e-9f20-cbe055136d9b-kube-api-access-57lzf\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.363549 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.363515 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-model-cache\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.363598 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.363551 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-home\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.363639 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.363601 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c9942b-32f5-491e-9f20-cbe055136d9b-tls-certs\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464435 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464400 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-dshm\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464552 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-tmp-dir\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464552 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57lzf\" (UniqueName: \"kubernetes.io/projected/45c9942b-32f5-491e-9f20-cbe055136d9b-kube-api-access-57lzf\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464552 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-model-cache\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464552 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464504 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-home\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464552 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464526 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c9942b-32f5-491e-9f20-cbe055136d9b-tls-certs\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.464943 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464917 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-tmp-dir\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.465019 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-model-cache\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.465019 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.464962 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-home\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.466609 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.466582 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-dshm\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.466880 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.466862 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c9942b-32f5-491e-9f20-cbe055136d9b-tls-certs\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.473253 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.473234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57lzf\" (UniqueName: \"kubernetes.io/projected/45c9942b-32f5-491e-9f20-cbe055136d9b-kube-api-access-57lzf\") pod \"test-model-faacd6-kserve-7c59cb87c9-2hwrv\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.651565 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.651502 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:43.784269 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.784244 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv"] Jul 10 13:41:43.786183 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:41:43.786154 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45c9942b_32f5_491e_9f20_cbe055136d9b.slice/crio-6ebba987ae941838a8db87ae42710c5a6209c67c8a3b426ebfb2951d0d2d12a2 WatchSource:0}: Error finding container 6ebba987ae941838a8db87ae42710c5a6209c67c8a3b426ebfb2951d0d2d12a2: Status 404 returned error can't find the container with id 6ebba987ae941838a8db87ae42710c5a6209c67c8a3b426ebfb2951d0d2d12a2 Jul 10 13:41:43.820653 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:43.820626 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" event={"ID":"45c9942b-32f5-491e-9f20-cbe055136d9b","Type":"ContainerStarted","Data":"6ebba987ae941838a8db87ae42710c5a6209c67c8a3b426ebfb2951d0d2d12a2"} Jul 10 13:41:44.827792 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:44.827760 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" event={"ID":"45c9942b-32f5-491e-9f20-cbe055136d9b","Type":"ContainerStarted","Data":"dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4"} Jul 10 13:41:44.844822 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:44.844777 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" podStartSLOduration=1.8447631599999998 podStartE2EDuration="1.84476316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:41:44.843770705 +0000 UTC m=+1884.565696583" watchObservedRunningTime="2026-07-10 13:41:44.84476316 +0000 UTC m=+1884.566689092" Jul 10 13:41:53.652065 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:53.652018 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:53.652483 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:53.652096 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:53.664530 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:53.664500 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:41:53.898900 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:41:53.898859 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:42:45.628941 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:42:45.628911 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv"] Jul 10 13:43:00.649187 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:00.649124 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" podUID="45c9942b-32f5-491e-9f20-cbe055136d9b" containerName="main" containerID="cri-o://dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4" gracePeriod=45 Jul 10 13:43:00.889502 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:00.889479 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:43:01.057320 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057294 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-model-cache\") pod \"45c9942b-32f5-491e-9f20-cbe055136d9b\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " Jul 10 13:43:01.057492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057331 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-home\") pod \"45c9942b-32f5-491e-9f20-cbe055136d9b\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " Jul 10 13:43:01.057492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057351 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-tmp-dir\") pod \"45c9942b-32f5-491e-9f20-cbe055136d9b\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " Jul 10 13:43:01.057492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057369 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57lzf\" (UniqueName: \"kubernetes.io/projected/45c9942b-32f5-491e-9f20-cbe055136d9b-kube-api-access-57lzf\") pod \"45c9942b-32f5-491e-9f20-cbe055136d9b\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " Jul 10 13:43:01.057492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057410 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-dshm\") pod \"45c9942b-32f5-491e-9f20-cbe055136d9b\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " Jul 10 13:43:01.057492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057482 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c9942b-32f5-491e-9f20-cbe055136d9b-tls-certs\") pod \"45c9942b-32f5-491e-9f20-cbe055136d9b\" (UID: \"45c9942b-32f5-491e-9f20-cbe055136d9b\") " Jul 10 13:43:01.057734 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057564 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "45c9942b-32f5-491e-9f20-cbe055136d9b" (UID: "45c9942b-32f5-491e-9f20-cbe055136d9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:43:01.057734 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057593 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-model-cache" (OuterVolumeSpecName: "model-cache") pod "45c9942b-32f5-491e-9f20-cbe055136d9b" (UID: "45c9942b-32f5-491e-9f20-cbe055136d9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:43:01.057734 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057612 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-home" (OuterVolumeSpecName: "home") pod "45c9942b-32f5-491e-9f20-cbe055136d9b" (UID: "45c9942b-32f5-491e-9f20-cbe055136d9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:43:01.057884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057791 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:01.057884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057805 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:01.057884 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.057832 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:01.059646 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.059611 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45c9942b-32f5-491e-9f20-cbe055136d9b-kube-api-access-57lzf" (OuterVolumeSpecName: "kube-api-access-57lzf") pod "45c9942b-32f5-491e-9f20-cbe055136d9b" (UID: "45c9942b-32f5-491e-9f20-cbe055136d9b"). InnerVolumeSpecName "kube-api-access-57lzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:43:01.059734 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.059655 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-dshm" (OuterVolumeSpecName: "dshm") pod "45c9942b-32f5-491e-9f20-cbe055136d9b" (UID: "45c9942b-32f5-491e-9f20-cbe055136d9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:43:01.059734 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.059656 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45c9942b-32f5-491e-9f20-cbe055136d9b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45c9942b-32f5-491e-9f20-cbe055136d9b" (UID: "45c9942b-32f5-491e-9f20-cbe055136d9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:43:01.159251 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.159225 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c9942b-32f5-491e-9f20-cbe055136d9b-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:01.159251 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.159248 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c9942b-32f5-491e-9f20-cbe055136d9b-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:01.159397 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.159259 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57lzf\" (UniqueName: \"kubernetes.io/projected/45c9942b-32f5-491e-9f20-cbe055136d9b-kube-api-access-57lzf\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:01.205473 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.205444 2571 generic.go:356] "Generic (PLEG): container finished" podID="45c9942b-32f5-491e-9f20-cbe055136d9b" containerID="dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4" exitCode=0 Jul 10 13:43:01.205580 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.205495 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" event={"ID":"45c9942b-32f5-491e-9f20-cbe055136d9b","Type":"ContainerDied","Data":"dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4"} Jul 10 13:43:01.205580 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.205523 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" event={"ID":"45c9942b-32f5-491e-9f20-cbe055136d9b","Type":"ContainerDied","Data":"6ebba987ae941838a8db87ae42710c5a6209c67c8a3b426ebfb2951d0d2d12a2"} Jul 10 13:43:01.205580 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.205539 2571 scope.go:117] "RemoveContainer" containerID="dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4" Jul 10 13:43:01.205580 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.205496 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv" Jul 10 13:43:01.214044 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.214027 2571 scope.go:117] "RemoveContainer" containerID="dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4" Jul 10 13:43:01.214302 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:43:01.214278 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4\": container with ID starting with dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4 not found: ID does not exist" containerID="dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4" Jul 10 13:43:01.214390 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.214307 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4"} err="failed to get container status \"dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4\": rpc error: code = NotFound desc = could not find container \"dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4\": container with ID starting with dd112ed7fdc1397fe6795f72dfa84cf20cf6bbe272736d04e34384f384a3c7d4 not found: ID does not exist" Jul 10 13:43:01.225732 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.225708 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv"] Jul 10 13:43:01.229857 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.229834 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-faacd6/test-model-faacd6-kserve-7c59cb87c9-2hwrv"] Jul 10 13:43:01.507988 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:01.507956 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45c9942b-32f5-491e-9f20-cbe055136d9b" path="/var/lib/kubelet/pods/45c9942b-32f5-491e-9f20-cbe055136d9b/volumes" Jul 10 13:43:29.884544 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:29.884511 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b"] Jul 10 13:43:29.884950 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:29.884721 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" podUID="cc1f9c99-91b1-4720-91a2-252fad40a755" containerName="maas-api" containerID="cri-o://776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56" gracePeriod=30 Jul 10 13:43:30.125746 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.125719 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:43:30.187536 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.187477 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vb8g\" (UniqueName: \"kubernetes.io/projected/cc1f9c99-91b1-4720-91a2-252fad40a755-kube-api-access-2vb8g\") pod \"cc1f9c99-91b1-4720-91a2-252fad40a755\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " Jul 10 13:43:30.187536 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.187518 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1f9c99-91b1-4720-91a2-252fad40a755-openshift-service-ca\") pod \"cc1f9c99-91b1-4720-91a2-252fad40a755\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " Jul 10 13:43:30.187731 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.187563 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc1f9c99-91b1-4720-91a2-252fad40a755-maas-api-tls\") pod \"cc1f9c99-91b1-4720-91a2-252fad40a755\" (UID: \"cc1f9c99-91b1-4720-91a2-252fad40a755\") " Jul 10 13:43:30.187952 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.187916 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc1f9c99-91b1-4720-91a2-252fad40a755-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc1f9c99-91b1-4720-91a2-252fad40a755" (UID: "cc1f9c99-91b1-4720-91a2-252fad40a755"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:43:30.189631 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.189604 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc1f9c99-91b1-4720-91a2-252fad40a755-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc1f9c99-91b1-4720-91a2-252fad40a755" (UID: "cc1f9c99-91b1-4720-91a2-252fad40a755"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:43:30.189719 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.189699 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc1f9c99-91b1-4720-91a2-252fad40a755-kube-api-access-2vb8g" (OuterVolumeSpecName: "kube-api-access-2vb8g") pod "cc1f9c99-91b1-4720-91a2-252fad40a755" (UID: "cc1f9c99-91b1-4720-91a2-252fad40a755"). InnerVolumeSpecName "kube-api-access-2vb8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:43:30.288603 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.288568 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc1f9c99-91b1-4720-91a2-252fad40a755-maas-api-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:30.288603 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.288601 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vb8g\" (UniqueName: \"kubernetes.io/projected/cc1f9c99-91b1-4720-91a2-252fad40a755-kube-api-access-2vb8g\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:30.288603 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.288611 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1f9c99-91b1-4720-91a2-252fad40a755-openshift-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:43:30.339988 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.339959 2571 generic.go:356] "Generic (PLEG): container finished" podID="cc1f9c99-91b1-4720-91a2-252fad40a755" containerID="776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56" exitCode=0 Jul 10 13:43:30.340110 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.340004 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" Jul 10 13:43:30.340110 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.340030 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" event={"ID":"cc1f9c99-91b1-4720-91a2-252fad40a755","Type":"ContainerDied","Data":"776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56"} Jul 10 13:43:30.340110 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.340097 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b" event={"ID":"cc1f9c99-91b1-4720-91a2-252fad40a755","Type":"ContainerDied","Data":"49560afdae984429a9c81aac55144fdc8e83b7167ce707aa8b982b168128b9e3"} Jul 10 13:43:30.340268 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.340131 2571 scope.go:117] "RemoveContainer" containerID="776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56" Jul 10 13:43:30.349333 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.349319 2571 scope.go:117] "RemoveContainer" containerID="776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56" Jul 10 13:43:30.349603 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:43:30.349584 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56\": container with ID starting with 776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56 not found: ID does not exist" containerID="776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56" Jul 10 13:43:30.349649 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.349612 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56"} err="failed to get container status \"776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56\": rpc error: code = NotFound desc = could not find container \"776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56\": container with ID starting with 776726819c64fcb10e16a775198615973ee44595133a246111c4c3a6fc054e56 not found: ID does not exist" Jul 10 13:43:30.361912 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.361890 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b"] Jul 10 13:43:30.367829 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:30.367808 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faacd6-84ff4f9ff4-cwl7b"] Jul 10 13:43:31.509195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:31.509163 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1f9c99-91b1-4720-91a2-252fad40a755" path="/var/lib/kubelet/pods/cc1f9c99-91b1-4720-91a2-252fad40a755/volumes" Jul 10 13:43:50.846363 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.846330 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz"] Jul 10 13:43:50.846900 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.846881 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc1f9c99-91b1-4720-91a2-252fad40a755" containerName="maas-api" Jul 10 13:43:50.846996 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.846908 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1f9c99-91b1-4720-91a2-252fad40a755" containerName="maas-api" Jul 10 13:43:50.846996 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.846925 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45c9942b-32f5-491e-9f20-cbe055136d9b" containerName="main" Jul 10 13:43:50.846996 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.846930 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="45c9942b-32f5-491e-9f20-cbe055136d9b" containerName="main" Jul 10 13:43:50.847172 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.846997 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc1f9c99-91b1-4720-91a2-252fad40a755" containerName="maas-api" Jul 10 13:43:50.847172 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.847005 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="45c9942b-32f5-491e-9f20-cbe055136d9b" containerName="main" Jul 10 13:43:50.851540 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.851517 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:50.854034 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.854016 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e693af-serving-cert\"" Jul 10 13:43:50.862601 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.862581 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz"] Jul 10 13:43:50.959333 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.959302 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa59066-d263-46e4-9613-3abc88626f7f-maas-api-tls\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:50.959486 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.959415 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/aaa59066-d263-46e4-9613-3abc88626f7f-kube-api-access-rnrcf\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:50.959486 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:50.959460 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa59066-d263-46e4-9613-3abc88626f7f-openshift-service-ca\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.060492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.060464 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/aaa59066-d263-46e4-9613-3abc88626f7f-kube-api-access-rnrcf\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.060629 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.060511 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa59066-d263-46e4-9613-3abc88626f7f-openshift-service-ca\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.060629 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.060553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa59066-d263-46e4-9613-3abc88626f7f-maas-api-tls\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.061175 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.061154 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa59066-d263-46e4-9613-3abc88626f7f-openshift-service-ca\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.062916 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.062896 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa59066-d263-46e4-9613-3abc88626f7f-maas-api-tls\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.076172 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.076150 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/aaa59066-d263-46e4-9613-3abc88626f7f-kube-api-access-rnrcf\") pod \"maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.163030 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.162941 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:51.316757 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.316721 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz"] Jul 10 13:43:51.319218 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:43:51.319191 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa59066_d263_46e4_9613_3abc88626f7f.slice/crio-80c222021b98c875dcd0834a84baaed01f175b9c659c3928bdfb1aadc0e5d2c4 WatchSource:0}: Error finding container 80c222021b98c875dcd0834a84baaed01f175b9c659c3928bdfb1aadc0e5d2c4: Status 404 returned error can't find the container with id 80c222021b98c875dcd0834a84baaed01f175b9c659c3928bdfb1aadc0e5d2c4 Jul 10 13:43:51.444342 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:51.444253 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" event={"ID":"aaa59066-d263-46e4-9613-3abc88626f7f","Type":"ContainerStarted","Data":"80c222021b98c875dcd0834a84baaed01f175b9c659c3928bdfb1aadc0e5d2c4"} Jul 10 13:43:52.450169 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:52.450130 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" event={"ID":"aaa59066-d263-46e4-9613-3abc88626f7f","Type":"ContainerStarted","Data":"f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb"} Jul 10 13:43:52.450548 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:52.450209 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:43:52.466497 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:52.466450 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" podStartSLOduration=2.17492032 podStartE2EDuration="2.466433263s" totalImagesPullingTime="291.512943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:43:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:43:52.46626701 +0000 UTC m=+2012.188192893" watchObservedRunningTime="2026-07-10 13:43:52.466433263 +0000 UTC m=+2012.188359145" Jul 10 13:43:58.459550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:43:58.459522 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:44:11.711124 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:11.711040 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz"] Jul 10 13:44:11.711860 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:11.711826 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" podUID="aaa59066-d263-46e4-9613-3abc88626f7f" containerName="maas-api" containerID="cri-o://f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb" gracePeriod=30 Jul 10 13:44:11.960588 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:11.960559 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:44:12.040833 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.040804 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa59066-d263-46e4-9613-3abc88626f7f-maas-api-tls\") pod \"aaa59066-d263-46e4-9613-3abc88626f7f\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " Jul 10 13:44:12.040976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.040842 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa59066-d263-46e4-9613-3abc88626f7f-openshift-service-ca\") pod \"aaa59066-d263-46e4-9613-3abc88626f7f\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " Jul 10 13:44:12.040976 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.040928 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/aaa59066-d263-46e4-9613-3abc88626f7f-kube-api-access-rnrcf\") pod \"aaa59066-d263-46e4-9613-3abc88626f7f\" (UID: \"aaa59066-d263-46e4-9613-3abc88626f7f\") " Jul 10 13:44:12.041280 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.041253 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa59066-d263-46e4-9613-3abc88626f7f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aaa59066-d263-46e4-9613-3abc88626f7f" (UID: "aaa59066-d263-46e4-9613-3abc88626f7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:44:12.042776 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.042756 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa59066-d263-46e4-9613-3abc88626f7f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aaa59066-d263-46e4-9613-3abc88626f7f" (UID: "aaa59066-d263-46e4-9613-3abc88626f7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:44:12.042895 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.042877 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa59066-d263-46e4-9613-3abc88626f7f-kube-api-access-rnrcf" (OuterVolumeSpecName: "kube-api-access-rnrcf") pod "aaa59066-d263-46e4-9613-3abc88626f7f" (UID: "aaa59066-d263-46e4-9613-3abc88626f7f"). InnerVolumeSpecName "kube-api-access-rnrcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:44:12.141939 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.141912 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/aaa59066-d263-46e4-9613-3abc88626f7f-kube-api-access-rnrcf\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:44:12.141939 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.141934 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa59066-d263-46e4-9613-3abc88626f7f-maas-api-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:44:12.142089 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.141944 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa59066-d263-46e4-9613-3abc88626f7f-openshift-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:44:12.542508 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.542476 2571 generic.go:356] "Generic (PLEG): container finished" podID="aaa59066-d263-46e4-9613-3abc88626f7f" containerID="f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb" exitCode=0 Jul 10 13:44:12.542508 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.542509 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" Jul 10 13:44:12.542756 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.542546 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" event={"ID":"aaa59066-d263-46e4-9613-3abc88626f7f","Type":"ContainerDied","Data":"f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb"} Jul 10 13:44:12.542756 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.542591 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz" event={"ID":"aaa59066-d263-46e4-9613-3abc88626f7f","Type":"ContainerDied","Data":"80c222021b98c875dcd0834a84baaed01f175b9c659c3928bdfb1aadc0e5d2c4"} Jul 10 13:44:12.542756 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.542621 2571 scope.go:117] "RemoveContainer" containerID="f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb" Jul 10 13:44:12.551927 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.551907 2571 scope.go:117] "RemoveContainer" containerID="f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb" Jul 10 13:44:12.552171 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:44:12.552155 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb\": container with ID starting with f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb not found: ID does not exist" containerID="f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb" Jul 10 13:44:12.552240 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.552177 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb"} err="failed to get container status \"f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb\": rpc error: code = NotFound desc = could not find container \"f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb\": container with ID starting with f7b37e42dcd45b392d07a383c62857061816cc042dcc1b437dcfc3ec596ffebb not found: ID does not exist" Jul 10 13:44:12.568225 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.568203 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz"] Jul 10 13:44:12.573501 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:12.573480 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e693af-96fbb85f6-hzxzz"] Jul 10 13:44:13.508149 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:13.508118 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa59066-d263-46e4-9613-3abc88626f7f" path="/var/lib/kubelet/pods/aaa59066-d263-46e4-9613-3abc88626f7f/volumes" Jul 10 13:44:30.140004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.139967 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn"] Jul 10 13:44:30.140499 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.140365 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa59066-d263-46e4-9613-3abc88626f7f" containerName="maas-api" Jul 10 13:44:30.140499 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.140378 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa59066-d263-46e4-9613-3abc88626f7f" containerName="maas-api" Jul 10 13:44:30.140499 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.140440 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa59066-d263-46e4-9613-3abc88626f7f" containerName="maas-api" Jul 10 13:44:30.143278 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.143261 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.149811 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.149686 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-efd2ad-serving-cert\"" Jul 10 13:44:30.169507 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.169485 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn"] Jul 10 13:44:30.200215 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.200192 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjq54\" (UniqueName: \"kubernetes.io/projected/98c3a636-7b75-42be-a5c3-deb24fd77ae0-kube-api-access-rjq54\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.200334 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.200240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c3a636-7b75-42be-a5c3-deb24fd77ae0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.200334 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.200307 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98c3a636-7b75-42be-a5c3-deb24fd77ae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.301268 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.301239 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c3a636-7b75-42be-a5c3-deb24fd77ae0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.301382 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.301290 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98c3a636-7b75-42be-a5c3-deb24fd77ae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.301382 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.301371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjq54\" (UniqueName: \"kubernetes.io/projected/98c3a636-7b75-42be-a5c3-deb24fd77ae0-kube-api-access-rjq54\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.301902 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.301877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c3a636-7b75-42be-a5c3-deb24fd77ae0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.303823 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.303798 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98c3a636-7b75-42be-a5c3-deb24fd77ae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.312284 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.312258 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjq54\" (UniqueName: \"kubernetes.io/projected/98c3a636-7b75-42be-a5c3-deb24fd77ae0-kube-api-access-rjq54\") pod \"maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.453155 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.453099 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:30.588580 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.588555 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn"] Jul 10 13:44:30.590878 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:44:30.590850 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c3a636_7b75_42be_a5c3_deb24fd77ae0.slice/crio-18c411d43d56db850bfadb86b2be01c904b397b54568cb1cb2a91357e904168e WatchSource:0}: Error finding container 18c411d43d56db850bfadb86b2be01c904b397b54568cb1cb2a91357e904168e: Status 404 returned error can't find the container with id 18c411d43d56db850bfadb86b2be01c904b397b54568cb1cb2a91357e904168e Jul 10 13:44:30.639881 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:30.639848 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" event={"ID":"98c3a636-7b75-42be-a5c3-deb24fd77ae0","Type":"ContainerStarted","Data":"18c411d43d56db850bfadb86b2be01c904b397b54568cb1cb2a91357e904168e"} Jul 10 13:44:31.646236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:31.646197 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" event={"ID":"98c3a636-7b75-42be-a5c3-deb24fd77ae0","Type":"ContainerStarted","Data":"f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044"} Jul 10 13:44:31.646659 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:31.646324 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:44:31.679874 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:31.679827 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" podStartSLOduration=1.296845464 podStartE2EDuration="1.679811957s" totalImagesPullingTime="382.966493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:44:31.677785251 +0000 UTC m=+2051.399711133" watchObservedRunningTime="2026-07-10 13:44:31.679811957 +0000 UTC m=+2051.401737841" Jul 10 13:44:37.656819 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:44:37.656788 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:45:21.516401 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:21.516366 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:45:21.521305 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:21.521285 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:45:31.437826 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.437792 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx"] Jul 10 13:45:31.441259 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.441239 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.444015 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.443990 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-efd2ad\"/\"kube-root-ca.crt\"" Jul 10 13:45:31.444947 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.444929 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-efd2ad\"/\"openshift-service-ca.crt\"" Jul 10 13:45:31.445034 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.444936 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-efd2ad\"/\"default-dockercfg-bnqhj\"" Jul 10 13:45:31.445136 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.444936 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-efd2ad\"/\"auth-test-model-efd2ad-kserve-self-signed-certs\"" Jul 10 13:45:31.454532 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.454512 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx"] Jul 10 13:45:31.531937 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.531903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-dshm\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.531937 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.531940 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjws9\" (UniqueName: \"kubernetes.io/projected/927b5365-04b4-4e37-9254-c9a93598ff5f-kube-api-access-xjws9\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.532196 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.531977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-model-cache\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.532196 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.532022 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-tmp-dir\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.532196 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.532097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-home\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.532196 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.532111 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/927b5365-04b4-4e37-9254-c9a93598ff5f-tls-certs\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633362 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633321 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-home\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633362 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/927b5365-04b4-4e37-9254-c9a93598ff5f-tls-certs\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633636 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633396 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-dshm\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633636 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjws9\" (UniqueName: \"kubernetes.io/projected/927b5365-04b4-4e37-9254-c9a93598ff5f-kube-api-access-xjws9\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633636 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633614 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-model-cache\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633797 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-tmp-dir\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.633851 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633807 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-home\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.634010 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.633982 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-model-cache\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.634101 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.634047 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-tmp-dir\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.635772 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.635749 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-dshm\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.635988 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.635968 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/927b5365-04b4-4e37-9254-c9a93598ff5f-tls-certs\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.645996 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.645971 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjws9\" (UniqueName: \"kubernetes.io/projected/927b5365-04b4-4e37-9254-c9a93598ff5f-kube-api-access-xjws9\") pod \"auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.751101 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.751041 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:31.899298 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.899258 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx"] Jul 10 13:45:31.902422 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:45:31.902393 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod927b5365_04b4_4e37_9254_c9a93598ff5f.slice/crio-a3ad12e22745ed165433b1657071fa96e04cdae4d7e8cb881a881d6a6d3671b8 WatchSource:0}: Error finding container a3ad12e22745ed165433b1657071fa96e04cdae4d7e8cb881a881d6a6d3671b8: Status 404 returned error can't find the container with id a3ad12e22745ed165433b1657071fa96e04cdae4d7e8cb881a881d6a6d3671b8 Jul 10 13:45:31.923475 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:31.923434 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" event={"ID":"927b5365-04b4-4e37-9254-c9a93598ff5f","Type":"ContainerStarted","Data":"a3ad12e22745ed165433b1657071fa96e04cdae4d7e8cb881a881d6a6d3671b8"} Jul 10 13:45:32.929404 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:32.929363 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" event={"ID":"927b5365-04b4-4e37-9254-c9a93598ff5f","Type":"ContainerStarted","Data":"08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003"} Jul 10 13:45:32.949544 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:32.949491 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" podStartSLOduration=1.949476136 podStartE2EDuration="1.949476136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:45:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:45:32.947676091 +0000 UTC m=+2112.669601997" watchObservedRunningTime="2026-07-10 13:45:32.949476136 +0000 UTC m=+2112.671402055" Jul 10 13:45:41.751567 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:41.751525 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:41.752047 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:41.751583 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:41.764333 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:41.764292 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:45:41.984375 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:45:41.984343 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:47:39.737004 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:39.736971 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx"] Jul 10 13:47:54.757195 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:54.757150 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" podUID="927b5365-04b4-4e37-9254-c9a93598ff5f" containerName="main" containerID="cri-o://08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003" gracePeriod=45 Jul 10 13:47:55.008136 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.008081 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:47:55.161214 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161186 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-model-cache\") pod \"927b5365-04b4-4e37-9254-c9a93598ff5f\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " Jul 10 13:47:55.161388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161228 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-dshm\") pod \"927b5365-04b4-4e37-9254-c9a93598ff5f\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " Jul 10 13:47:55.161388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161247 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjws9\" (UniqueName: \"kubernetes.io/projected/927b5365-04b4-4e37-9254-c9a93598ff5f-kube-api-access-xjws9\") pod \"927b5365-04b4-4e37-9254-c9a93598ff5f\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " Jul 10 13:47:55.161388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161328 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-tmp-dir\") pod \"927b5365-04b4-4e37-9254-c9a93598ff5f\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " Jul 10 13:47:55.161388 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161351 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/927b5365-04b4-4e37-9254-c9a93598ff5f-tls-certs\") pod \"927b5365-04b4-4e37-9254-c9a93598ff5f\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " Jul 10 13:47:55.161556 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161400 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-home\") pod \"927b5365-04b4-4e37-9254-c9a93598ff5f\" (UID: \"927b5365-04b4-4e37-9254-c9a93598ff5f\") " Jul 10 13:47:55.161556 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161458 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "927b5365-04b4-4e37-9254-c9a93598ff5f" (UID: "927b5365-04b4-4e37-9254-c9a93598ff5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:47:55.161556 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161547 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "927b5365-04b4-4e37-9254-c9a93598ff5f" (UID: "927b5365-04b4-4e37-9254-c9a93598ff5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:47:55.161692 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161672 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-home" (OuterVolumeSpecName: "home") pod "927b5365-04b4-4e37-9254-c9a93598ff5f" (UID: "927b5365-04b4-4e37-9254-c9a93598ff5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:47:55.161742 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161715 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:47:55.161742 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.161730 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:47:55.163663 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.163641 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/927b5365-04b4-4e37-9254-c9a93598ff5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "927b5365-04b4-4e37-9254-c9a93598ff5f" (UID: "927b5365-04b4-4e37-9254-c9a93598ff5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:47:55.163854 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.163830 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/927b5365-04b4-4e37-9254-c9a93598ff5f-kube-api-access-xjws9" (OuterVolumeSpecName: "kube-api-access-xjws9") pod "927b5365-04b4-4e37-9254-c9a93598ff5f" (UID: "927b5365-04b4-4e37-9254-c9a93598ff5f"). InnerVolumeSpecName "kube-api-access-xjws9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:47:55.163854 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.163834 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-dshm" (OuterVolumeSpecName: "dshm") pod "927b5365-04b4-4e37-9254-c9a93598ff5f" (UID: "927b5365-04b4-4e37-9254-c9a93598ff5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:47:55.263117 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.263026 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/927b5365-04b4-4e37-9254-c9a93598ff5f-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:47:55.263117 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.263075 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:47:55.263117 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.263083 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/927b5365-04b4-4e37-9254-c9a93598ff5f-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:47:55.263117 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.263091 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjws9\" (UniqueName: \"kubernetes.io/projected/927b5365-04b4-4e37-9254-c9a93598ff5f-kube-api-access-xjws9\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:47:55.599082 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.599033 2571 generic.go:356] "Generic (PLEG): container finished" podID="927b5365-04b4-4e37-9254-c9a93598ff5f" containerID="08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003" exitCode=0 Jul 10 13:47:55.599275 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.599097 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" Jul 10 13:47:55.599275 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.599122 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" event={"ID":"927b5365-04b4-4e37-9254-c9a93598ff5f","Type":"ContainerDied","Data":"08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003"} Jul 10 13:47:55.599275 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.599165 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx" event={"ID":"927b5365-04b4-4e37-9254-c9a93598ff5f","Type":"ContainerDied","Data":"a3ad12e22745ed165433b1657071fa96e04cdae4d7e8cb881a881d6a6d3671b8"} Jul 10 13:47:55.599275 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.599183 2571 scope.go:117] "RemoveContainer" containerID="08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003" Jul 10 13:47:55.607969 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.607944 2571 scope.go:117] "RemoveContainer" containerID="08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003" Jul 10 13:47:55.608222 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:47:55.608203 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003\": container with ID starting with 08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003 not found: ID does not exist" containerID="08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003" Jul 10 13:47:55.608284 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.608230 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003"} err="failed to get container status \"08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003\": rpc error: code = NotFound desc = could not find container \"08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003\": container with ID starting with 08b8b932e3d1b1e2ec0c1ae9049bde8b9f1f98be34be8d89173441f310c38003 not found: ID does not exist" Jul 10 13:47:55.614782 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.614758 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx"] Jul 10 13:47:55.619149 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:55.619127 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-efd2ad/auth-test-model-efd2ad-kserve-7fcdc749ff-l9lfx"] Jul 10 13:47:57.508271 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:47:57.508238 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="927b5365-04b4-4e37-9254-c9a93598ff5f" path="/var/lib/kubelet/pods/927b5365-04b4-4e37-9254-c9a93598ff5f/volumes" Jul 10 13:50:21.544160 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:50:21.544041 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:50:21.550708 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:50:21.550690 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b44jh_20158f5f-220b-47fe-9bd2-7099859065a6/ovn-acl-logging/0.log" Jul 10 13:51:40.596478 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.596441 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn"] Jul 10 13:51:40.597101 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.596682 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" podUID="98c3a636-7b75-42be-a5c3-deb24fd77ae0" containerName="maas-api" containerID="cri-o://f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044" gracePeriod=30 Jul 10 13:51:40.835461 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.835432 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:51:40.917213 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.917137 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c3a636-7b75-42be-a5c3-deb24fd77ae0-openshift-service-ca\") pod \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " Jul 10 13:51:40.917213 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.917192 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjq54\" (UniqueName: \"kubernetes.io/projected/98c3a636-7b75-42be-a5c3-deb24fd77ae0-kube-api-access-rjq54\") pod \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " Jul 10 13:51:40.917414 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.917359 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98c3a636-7b75-42be-a5c3-deb24fd77ae0-maas-api-tls\") pod \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\" (UID: \"98c3a636-7b75-42be-a5c3-deb24fd77ae0\") " Jul 10 13:51:40.917490 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.917465 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c3a636-7b75-42be-a5c3-deb24fd77ae0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98c3a636-7b75-42be-a5c3-deb24fd77ae0" (UID: "98c3a636-7b75-42be-a5c3-deb24fd77ae0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:51:40.917636 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.917619 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c3a636-7b75-42be-a5c3-deb24fd77ae0-openshift-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:51:40.919209 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.919179 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c3a636-7b75-42be-a5c3-deb24fd77ae0-kube-api-access-rjq54" (OuterVolumeSpecName: "kube-api-access-rjq54") pod "98c3a636-7b75-42be-a5c3-deb24fd77ae0" (UID: "98c3a636-7b75-42be-a5c3-deb24fd77ae0"). InnerVolumeSpecName "kube-api-access-rjq54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:51:40.919345 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:40.919326 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98c3a636-7b75-42be-a5c3-deb24fd77ae0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98c3a636-7b75-42be-a5c3-deb24fd77ae0" (UID: "98c3a636-7b75-42be-a5c3-deb24fd77ae0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:51:41.018689 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.018662 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98c3a636-7b75-42be-a5c3-deb24fd77ae0-maas-api-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:51:41.018689 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.018684 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjq54\" (UniqueName: \"kubernetes.io/projected/98c3a636-7b75-42be-a5c3-deb24fd77ae0-kube-api-access-rjq54\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\"" Jul 10 13:51:41.634144 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.634105 2571 generic.go:356] "Generic (PLEG): container finished" podID="98c3a636-7b75-42be-a5c3-deb24fd77ae0" containerID="f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044" exitCode=0 Jul 10 13:51:41.634496 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.634165 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" Jul 10 13:51:41.634496 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.634191 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" event={"ID":"98c3a636-7b75-42be-a5c3-deb24fd77ae0","Type":"ContainerDied","Data":"f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044"} Jul 10 13:51:41.634496 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.634261 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn" event={"ID":"98c3a636-7b75-42be-a5c3-deb24fd77ae0","Type":"ContainerDied","Data":"18c411d43d56db850bfadb86b2be01c904b397b54568cb1cb2a91357e904168e"} Jul 10 13:51:41.634496 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.634282 2571 scope.go:117] "RemoveContainer" containerID="f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044" Jul 10 13:51:41.642403 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.642385 2571 scope.go:117] "RemoveContainer" containerID="f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044" Jul 10 13:51:41.642664 ip-10-0-133-196 kubenswrapper[2571]: E0710 13:51:41.642647 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044\": container with ID starting with f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044 not found: ID does not exist" containerID="f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044" Jul 10 13:51:41.642725 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.642674 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044"} err="failed to get container status \"f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044\": rpc error: code = NotFound desc = could not find container \"f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044\": container with ID starting with f5a6c996eba2ca9b109f485458a886c97d6ca51f51675c45f6dbd26574daa044 not found: ID does not exist" Jul 10 13:51:41.649480 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.649450 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn"] Jul 10 13:51:41.653968 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:41.653947 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-efd2ad-566bcb4bc7-vqtxn"] Jul 10 13:51:43.508837 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:43.508808 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c3a636-7b75-42be-a5c3-deb24fd77ae0" path="/var/lib/kubelet/pods/98c3a636-7b75-42be-a5c3-deb24fd77ae0/volumes" Jul 10 13:51:47.326492 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:47.326462 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f5b9bf496-5wggg_510fbbbc-ceb9-4c54-a3fa-35485a2129fe/authorino/0.log" Jul 10 13:51:52.279702 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:52.279665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-54zm6_f0cb099f-958a-4f33-849d-ff079e918f13/manager/0.log" Jul 10 13:51:52.404014 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:52.403983 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-668b649c84-jkhpt_b93ef924-abcd-4a3c-a5e9-a8e228484c1b/manager/0.log" Jul 10 13:51:52.996226 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:52.996194 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ff96f8f8-w5xw2_6201d837-2dc1-490b-b067-f5780e4d61eb/copy-manifests/0.log" Jul 10 13:51:53.002185 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:53.002156 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ff96f8f8-w5xw2_6201d837-2dc1-490b-b067-f5780e4d61eb/manager/0.log" Jul 10 13:51:53.318550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:53.318471 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-b74766796-gj6d7_afe04d0e-0360-4e86-9236-5cc468e624b3/maas-api/0.log" Jul 10 13:51:54.731442 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:54.731412 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f5b9bf496-5wggg_510fbbbc-ceb9-4c54-a3fa-35485a2129fe/authorino/0.log" Jul 10 13:51:56.282450 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:56.282415 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f6cf8c47-rrg7z_c06b0362-393f-459c-9bba-9d5f2bc9ebc7/kube-auth-proxy/0.log" Jul 10 13:51:56.515612 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:56.515581 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-54f59_cab35f80-a85b-4b19-935c-ef4450e3f8a8/payload-pre-processing/0.log" Jul 10 13:51:56.632775 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:56.632701 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nvw74_2fd2f80e-6219-4b37-8d41-824bc12a9637/payload-processing/0.log" Jul 10 13:51:57.200858 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:57.200820 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wqr2t_83b80762-8628-426c-9144-e560c8930e71/main/0.log" Jul 10 13:51:57.413073 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:57.413029 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6pjg_9ca4b5cb-7f0a-4f8e-87be-af78ad55e8c0/main/0.log" Jul 10 13:51:57.514961 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:57.514933 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-jg7js_7407534a-a843-4aa9-918e-06693d4b5f62/main/0.log" Jul 10 13:51:57.630481 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:51:57.630458 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pr4fn_28b3cedd-db76-4446-8243-85699e42a39d/main/0.log" Jul 10 13:52:04.504584 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:04.504556 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xl7w9_0b05fa0c-f2b6-440b-950f-1d316453a43c/global-pull-secret-syncer/0.log" Jul 10 13:52:04.601772 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:04.601739 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q2cr9_881293b5-931f-4d9e-a312-29e17abf5c52/konnectivity-agent/0.log" Jul 10 13:52:04.696780 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:04.696752 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-196.ec2.internal_2cf7db545b083efad4fe99dd4fc40cef/haproxy/0.log" Jul 10 13:52:09.955456 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:09.955422 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f5b9bf496-5wggg_510fbbbc-ceb9-4c54-a3fa-35485a2129fe/authorino/0.log" Jul 10 13:52:12.024949 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.024912 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/alertmanager/0.log" Jul 10 13:52:12.053960 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.053932 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/config-reloader/0.log" Jul 10 13:52:12.095833 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.095765 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/kube-rbac-proxy-web/0.log" Jul 10 13:52:12.121329 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.121297 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/kube-rbac-proxy/0.log" Jul 10 13:52:12.147868 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.147841 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/kube-rbac-proxy-metric/0.log" Jul 10 13:52:12.169750 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.169721 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/prom-label-proxy/0.log" Jul 10 13:52:12.197091 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.197043 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e15db714-8bcb-4279-b35a-b9c11f7da3fb/init-config-reloader/0.log" Jul 10 13:52:12.729008 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.728924 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zmtdl_32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c/node-exporter/0.log" Jul 10 13:52:12.753995 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.753970 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zmtdl_32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c/kube-rbac-proxy/0.log" Jul 10 13:52:12.777275 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.777255 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zmtdl_32176f3e-0c0d-4c44-8c8c-1ab317b9bf1c/init-textfile/0.log" Jul 10 13:52:12.805236 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.805214 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tqzjv_654c2dad-f0a1-4445-bf21-fc93d5940827/kube-rbac-proxy-main/0.log" Jul 10 13:52:12.828268 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.828248 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tqzjv_654c2dad-f0a1-4445-bf21-fc93d5940827/kube-rbac-proxy-self/0.log" Jul 10 13:52:12.852762 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.852744 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tqzjv_654c2dad-f0a1-4445-bf21-fc93d5940827/openshift-state-metrics/0.log" Jul 10 13:52:12.891605 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.891590 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/prometheus/0.log" Jul 10 13:52:12.915930 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.915907 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/config-reloader/0.log" Jul 10 13:52:12.940272 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.940252 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/thanos-sidecar/0.log" Jul 10 13:52:12.964046 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:12.964029 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/kube-rbac-proxy-web/0.log" Jul 10 13:52:13.005529 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:13.005504 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/kube-rbac-proxy/0.log" Jul 10 13:52:13.028921 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:13.028896 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/kube-rbac-proxy-thanos/0.log" Jul 10 13:52:13.057239 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:13.057218 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddc2a65e-6b13-4bba-880b-ac49b46b7433/init-config-reloader/0.log" Jul 10 13:52:13.097186 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:13.097156 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-d5dk8_24f8e8a1-609f-4581-ad06-8cb02f7fbd55/prometheus-operator/0.log" Jul 10 13:52:13.117848 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:13.117817 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-d5dk8_24f8e8a1-609f-4581-ad06-8cb02f7fbd55/kube-rbac-proxy/0.log" Jul 10 13:52:14.558409 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558376 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p"] Jul 10 13:52:14.558795 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558731 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c3a636-7b75-42be-a5c3-deb24fd77ae0" containerName="maas-api" Jul 10 13:52:14.558795 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558741 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c3a636-7b75-42be-a5c3-deb24fd77ae0" containerName="maas-api" Jul 10 13:52:14.558795 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558753 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="927b5365-04b4-4e37-9254-c9a93598ff5f" containerName="main" Jul 10 13:52:14.558795 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558759 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="927b5365-04b4-4e37-9254-c9a93598ff5f" containerName="main" Jul 10 13:52:14.558947 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558808 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c3a636-7b75-42be-a5c3-deb24fd77ae0" containerName="maas-api" Jul 10 13:52:14.558947 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.558821 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="927b5365-04b4-4e37-9254-c9a93598ff5f" containerName="main" Jul 10 13:52:14.561655 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.561639 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.564295 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.564268 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k76w9\"/\"kube-root-ca.crt\"" Jul 10 13:52:14.564429 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.564413 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k76w9\"/\"openshift-service-ca.crt\"" Jul 10 13:52:14.565264 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.565250 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k76w9\"/\"default-dockercfg-wps8p\"" Jul 10 13:52:14.571011 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.570991 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p"] Jul 10 13:52:14.613078 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.613023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-sys\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.613239 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.613086 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-lib-modules\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.613239 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.613133 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-proc\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.613239 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.613199 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7z6c\" (UniqueName: \"kubernetes.io/projected/7a45006b-504e-4fab-9504-1941031fd670-kube-api-access-d7z6c\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.613239 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.613237 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-podres\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714183 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714145 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-sys\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714328 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714188 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-lib-modules\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714328 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714219 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-proc\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714328 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714269 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7z6c\" (UniqueName: \"kubernetes.io/projected/7a45006b-504e-4fab-9504-1941031fd670-kube-api-access-d7z6c\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714328 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714279 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-sys\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714751 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714356 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-proc\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714751 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714388 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-podres\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714751 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714393 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-lib-modules\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.714751 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.714494 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7a45006b-504e-4fab-9504-1941031fd670-podres\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.722929 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.722904 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7z6c\" (UniqueName: \"kubernetes.io/projected/7a45006b-504e-4fab-9504-1941031fd670-kube-api-access-d7z6c\") pod \"perf-node-gather-daemonset-crj6p\" (UID: \"7a45006b-504e-4fab-9504-1941031fd670\") " pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:14.872288 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:14.872200 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:15.003870 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:15.003846 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p"] Jul 10 13:52:15.006181 ip-10-0-133-196 kubenswrapper[2571]: W0710 13:52:15.006156 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7a45006b_504e_4fab_9504_1941031fd670.slice/crio-60d2c3326a907ab4a920cbf87005fd686840d8492f7b3669ee9a907febf8e2d1 WatchSource:0}: Error finding container 60d2c3326a907ab4a920cbf87005fd686840d8492f7b3669ee9a907febf8e2d1: Status 404 returned error can't find the container with id 60d2c3326a907ab4a920cbf87005fd686840d8492f7b3669ee9a907febf8e2d1 Jul 10 13:52:15.637955 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:15.637928 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7575c499b6-ttgkq_a4a4f81d-ced2-4098-afd4-2f8d84f8f72e/console/0.log" Jul 10 13:52:15.793205 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:15.793167 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" event={"ID":"7a45006b-504e-4fab-9504-1941031fd670","Type":"ContainerStarted","Data":"d647f65652d39cb712891c9509cfe59d62abade9fcfa69187908a5de842032ba"} Jul 10 13:52:15.793368 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:15.793222 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" event={"ID":"7a45006b-504e-4fab-9504-1941031fd670","Type":"ContainerStarted","Data":"60d2c3326a907ab4a920cbf87005fd686840d8492f7b3669ee9a907febf8e2d1"} Jul 10 13:52:15.793368 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:15.793350 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:15.813110 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:15.813043 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" podStartSLOduration=1.8130285000000002 podStartE2EDuration="1.8130285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:52:15.811477415 +0000 UTC m=+2515.533403295" watchObservedRunningTime="2026-07-10 13:52:15.8130285 +0000 UTC m=+2515.534954382" Jul 10 13:52:17.418170 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:17.418135 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pfnk7_5961e326-2362-41a5-84cb-0eb37fb30150/dns/0.log" Jul 10 13:52:17.445364 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:17.445320 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pfnk7_5961e326-2362-41a5-84cb-0eb37fb30150/kube-rbac-proxy/0.log" Jul 10 13:52:17.581274 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:17.581219 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rnmbh_8530fac2-8e52-489b-99bb-ea51d491256c/dns-node-resolver/0.log" Jul 10 13:52:18.211164 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:18.211136 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mv57f_c5904000-3a60-4cea-80c3-85a812a39930/node-ca/0.log" Jul 10 13:52:19.477413 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:19.477380 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f6cf8c47-rrg7z_c06b0362-393f-459c-9bba-9d5f2bc9ebc7/kube-auth-proxy/0.log" Jul 10 13:52:19.606361 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:19.606332 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-54f59_cab35f80-a85b-4b19-935c-ef4450e3f8a8/payload-pre-processing/0.log" Jul 10 13:52:19.655957 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:19.655931 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nvw74_2fd2f80e-6219-4b37-8d41-824bc12a9637/payload-processing/0.log" Jul 10 13:52:20.537021 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:20.536990 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nmczh_f08d0d3e-3c8d-4eb3-8eef-786babdb96e7/serve-healthcheck-canary/0.log" Jul 10 13:52:21.073464 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:21.073435 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dkrrp_7774828c-22a9-4793-8947-7a36a29db1a8/kube-rbac-proxy/0.log" Jul 10 13:52:21.098636 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:21.098606 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dkrrp_7774828c-22a9-4793-8947-7a36a29db1a8/exporter/0.log" Jul 10 13:52:21.129477 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:21.129450 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dkrrp_7774828c-22a9-4793-8947-7a36a29db1a8/extractor/0.log" Jul 10 13:52:21.806703 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:21.806671 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k76w9/perf-node-gather-daemonset-crj6p" Jul 10 13:52:23.765872 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:23.765842 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-54zm6_f0cb099f-958a-4f33-849d-ff079e918f13/manager/0.log" Jul 10 13:52:23.859851 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:23.859778 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-668b649c84-jkhpt_b93ef924-abcd-4a3c-a5e9-a8e228484c1b/manager/0.log" Jul 10 13:52:24.273840 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:24.273816 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ff96f8f8-w5xw2_6201d837-2dc1-490b-b067-f5780e4d61eb/manager/0.log" Jul 10 13:52:24.304080 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:24.303772 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ff96f8f8-w5xw2_6201d837-2dc1-490b-b067-f5780e4d61eb/copy-manifests/0.log" Jul 10 13:52:25.338530 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:25.338499 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f6cf8c47-rrg7z_c06b0362-393f-459c-9bba-9d5f2bc9ebc7/kube-auth-proxy/0.log" Jul 10 13:52:25.445798 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:25.445770 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-54f59_cab35f80-a85b-4b19-935c-ef4450e3f8a8/payload-pre-processing/0.log" Jul 10 13:52:25.464501 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:25.464476 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nvw74_2fd2f80e-6219-4b37-8d41-824bc12a9637/payload-processing/0.log" Jul 10 13:52:25.530835 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:25.530807 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cffcb8bcf-9tktk_356252e9-2206-4ea6-a7fd-bc87d4f527b9/manager/0.log" Jul 10 13:52:26.903885 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:26.903859 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f6cf8c47-rrg7z_c06b0362-393f-459c-9bba-9d5f2bc9ebc7/kube-auth-proxy/0.log" Jul 10 13:52:27.003559 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:27.003533 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-54f59_cab35f80-a85b-4b19-935c-ef4450e3f8a8/payload-pre-processing/0.log" Jul 10 13:52:27.021362 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:27.021338 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nvw74_2fd2f80e-6219-4b37-8d41-824bc12a9637/payload-processing/0.log" Jul 10 13:52:31.914550 ip-10-0-133-196 kubenswrapper[2571]: I0710 13:52:31.914519 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5wbm9_706332e4-b03e-48e4-b23e-32aa43dac6f1/kube-multus/0.log"