Jul 06 18:15:35.783601 ip-10-0-133-250 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 18:15:35.783613 ip-10-0-133-250 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 18:15:35.783623 ip-10-0-133-250 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 18:15:35.783940 ip-10-0-133-250 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 18:15:46.032911 ip-10-0-133-250 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 18:15:46.032927 ip-10-0-133-250 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f8170f60327f46aab3a6868e7618d691 -- Jul 06 18:17:48.739713 ip-10-0-133-250 systemd[1]: Starting Kubernetes Kubelet... Jul 06 18:17:49.133025 ip-10-0-133-250 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:17:49.133025 ip-10-0-133-250 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 18:17:49.133025 ip-10-0-133-250 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:17:49.133025 ip-10-0-133-250 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 18:17:49.133025 ip-10-0-133-250 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:17:49.133942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.133850 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 18:17:49.138656 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138636 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:17:49.138656 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138655 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138659 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138664 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138669 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138673 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138677 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138680 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138683 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138686 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138689 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138691 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138694 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138697 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138700 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138703 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138705 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138708 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138711 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138714 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138725 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:17:49.138732 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138729 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138731 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138734 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138737 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138740 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138743 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138746 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138749 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138751 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138754 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138757 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138760 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138762 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138765 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138767 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138770 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138773 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138775 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138778 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138780 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:17:49.139270 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138783 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138786 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138789 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138791 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138794 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138797 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138800 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138803 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138805 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138808 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138811 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138813 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138816 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138822 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138825 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138830 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138835 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138838 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138841 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138844 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:17:49.139810 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138847 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138850 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138878 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138881 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138885 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138888 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138891 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138894 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138897 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138900 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138903 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138906 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138910 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138912 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138915 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138918 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138921 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138923 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138927 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:17:49.140349 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138930 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138934 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138938 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138941 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138944 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138948 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138951 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138954 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138958 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138961 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138964 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138967 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138970 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138973 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138975 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138979 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138983 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138986 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.138989 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:17:49.140869 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140571 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140580 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140585 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140589 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140593 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140598 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140603 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140606 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140610 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140613 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140616 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140619 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140622 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140625 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140637 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140640 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140643 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140646 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140649 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:17:49.141417 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140652 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140656 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140660 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140663 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140666 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140669 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140672 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140675 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140677 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140680 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140685 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140689 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140691 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140695 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140698 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140701 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140704 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140706 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140709 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140712 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:17:49.141945 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140715 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140718 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140721 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140724 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140727 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140730 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140733 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140736 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140738 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140741 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140744 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140747 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140751 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140755 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140758 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140760 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140763 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140766 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140769 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:17:49.142498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140771 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140774 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140777 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140780 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140782 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140785 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140787 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140790 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140794 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140797 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140800 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140802 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140805 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140808 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140811 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140814 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140816 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140819 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140822 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140826 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:17:49.142992 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140830 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140833 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140836 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140839 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140841 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140845 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140848 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140851 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140854 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140856 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140859 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140862 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140864 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140867 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140870 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140873 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140876 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140879 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140882 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140884 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:17:49.143518 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.140888 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141334 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141340 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141343 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141347 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141350 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141353 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141356 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141359 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141362 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141365 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141368 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141371 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141373 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141377 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141382 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141385 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141388 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141406 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:17:49.144044 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141409 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141412 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141417 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141421 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141425 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141428 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141431 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141436 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141439 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141442 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141445 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141448 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141451 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141454 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141457 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141460 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141463 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141466 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141469 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:17:49.144555 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141473 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141475 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141478 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141481 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141484 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141487 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141489 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141492 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141495 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141498 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141500 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141504 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141506 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141509 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141512 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141515 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141518 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141520 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141523 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141526 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:17:49.145185 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141528 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141531 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141535 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141539 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141541 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141544 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141547 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141549 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141552 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141556 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141560 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141563 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141565 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141569 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141572 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141574 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141577 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141580 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141582 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:17:49.145717 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141585 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141587 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141590 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141593 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141596 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141599 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141602 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141605 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141608 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141610 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141613 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141616 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141619 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141621 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141625 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141627 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141630 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141633 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141635 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141638 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:17:49.146207 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141640 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141643 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.141646 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141720 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141733 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141741 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141746 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141751 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141755 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141760 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141765 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141768 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141771 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141775 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141778 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141781 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141787 2566 flags.go:64] FLAG: --cgroup-root="" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141790 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141794 2566 flags.go:64] FLAG: --client-ca-file="" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141797 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141800 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141804 2566 flags.go:64] FLAG: --cluster-domain="" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141807 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141811 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 18:17:49.146731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141814 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141818 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141822 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141825 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141828 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141832 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141835 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141838 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141841 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141845 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141849 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141853 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141857 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141859 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141862 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141866 2566 flags.go:64] FLAG: --enable-server="true" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141871 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141876 2566 flags.go:64] FLAG: --event-burst="100" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141879 2566 flags.go:64] FLAG: --event-qps="50" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141882 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141886 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141889 2566 flags.go:64] FLAG: --eviction-hard="" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141893 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141896 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141903 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 18:17:49.147342 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141906 2566 flags.go:64] FLAG: --eviction-soft="" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141909 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141912 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141915 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141918 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141922 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141925 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141928 2566 flags.go:64] FLAG: --feature-gates="" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141932 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141936 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141939 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141943 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141946 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141949 2566 flags.go:64] FLAG: --help="false" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141953 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141956 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141959 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141963 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141966 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141969 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141972 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141975 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141978 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 18:17:49.148038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141981 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141985 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141988 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141991 2566 flags.go:64] FLAG: --kube-reserved="" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141995 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.141998 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142001 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142004 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142009 2566 flags.go:64] FLAG: --lock-file="" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142012 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142015 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142018 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142024 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142027 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142030 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142033 2566 flags.go:64] FLAG: --logging-format="text" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142036 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142040 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142043 2566 flags.go:64] FLAG: --manifest-url="" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142046 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142051 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142054 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142059 2566 flags.go:64] FLAG: --max-pods="110" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142063 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142066 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 18:17:49.148690 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142069 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142072 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142075 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142078 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142082 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142089 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142092 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142095 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142099 2566 flags.go:64] FLAG: --pod-cidr="" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142108 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142115 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142118 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142122 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142125 2566 flags.go:64] FLAG: --port="10250" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142128 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142132 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08ad6c7804eda4e22" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142137 2566 flags.go:64] FLAG: --qos-reserved="" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142140 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142143 2566 flags.go:64] FLAG: --register-node="true" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142146 2566 flags.go:64] FLAG: --register-with-taints="" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142150 2566 flags.go:64] FLAG: --registry-burst="10" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142153 2566 flags.go:64] FLAG: --registry-qps="5" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142156 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142159 2566 flags.go:64] FLAG: --reserved-memory="" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142165 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 18:17:49.149415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142168 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142171 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142175 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142177 2566 flags.go:64] FLAG: --runonce="false" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142181 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142184 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142188 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142191 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142194 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142197 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142200 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142204 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142207 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142210 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142213 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142216 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142220 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142224 2566 flags.go:64] FLAG: --system-cgroups="" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142227 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142234 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142236 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142240 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142244 2566 flags.go:64] FLAG: --tls-min-version="" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142249 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142252 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 18:17:49.150083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142255 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142258 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142262 2566 flags.go:64] FLAG: --v="2" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142266 2566 flags.go:64] FLAG: --version="false" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142271 2566 flags.go:64] FLAG: --vmodule="" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142275 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.142280 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142432 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142437 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142440 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142443 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142447 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142450 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142453 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142456 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142460 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142464 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142468 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142472 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142475 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:17:49.150749 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142478 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142481 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142483 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142487 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142489 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142493 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142496 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142499 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142502 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142505 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142510 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142513 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142516 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142519 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142522 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142524 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142527 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142530 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142534 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142537 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:17:49.151275 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142540 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142543 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142546 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142548 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142551 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142554 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142557 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142560 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142562 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142565 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142568 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142570 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142573 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142576 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142578 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142581 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142584 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142587 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142596 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142599 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:17:49.151866 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142601 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142605 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142608 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142611 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142613 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142616 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142619 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142621 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142624 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142627 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142631 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142634 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142637 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142639 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142642 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142645 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142647 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142650 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142653 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142655 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:17:49.152413 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142658 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142660 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142663 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142666 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142668 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142671 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142674 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142676 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142679 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142681 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142684 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142687 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142690 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142693 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142695 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142698 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142700 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142705 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142708 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142711 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:17:49.152974 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142714 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142717 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142724 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142727 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142730 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.142733 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.143467 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.151657 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.151679 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.151700 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.151706 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151790 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151796 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151800 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:17:49.153843 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151803 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151807 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151811 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151817 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151820 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151823 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151827 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151830 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151833 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151836 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151839 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151842 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151846 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151849 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151852 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151855 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151858 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151861 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151864 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:17:49.154438 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151867 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151870 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151873 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151875 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151879 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151882 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151884 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151895 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151898 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151901 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151905 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151907 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151910 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151913 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151916 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151919 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151922 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151924 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151927 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151930 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:17:49.155008 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151932 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151935 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151938 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151941 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151944 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151947 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151950 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151953 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151955 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151958 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151961 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151964 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151967 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151970 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151973 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151976 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151978 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151981 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151984 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151986 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:17:49.155629 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151995 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.151998 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152001 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152004 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152007 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152009 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152012 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152016 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152018 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152021 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152024 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152026 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152029 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152032 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152034 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152037 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152040 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152043 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152046 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:17:49.156152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152049 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152052 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152055 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152058 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152061 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152065 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152069 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152073 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152076 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152079 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152081 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152085 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152089 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152092 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152104 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152108 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152110 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:17:49.156670 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152113 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.152119 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152337 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152344 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152347 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152351 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152354 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152357 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152360 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152363 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152366 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152369 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152372 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152375 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152378 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:17:49.157115 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152381 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152384 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152386 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152389 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152407 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152412 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152414 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152417 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152420 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152424 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152428 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152432 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152435 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152437 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152440 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152450 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152453 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152456 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152459 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:17:49.157549 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152462 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152465 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152469 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152472 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152476 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152479 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152482 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152485 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152488 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152491 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152494 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152497 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152500 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152517 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152520 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152524 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152527 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152530 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152533 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152537 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:17:49.158048 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152540 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152543 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152546 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152549 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152552 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152555 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152558 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152561 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152564 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152573 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152576 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152579 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152581 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152584 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152587 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152589 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152592 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152595 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152597 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152600 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:17:49.158597 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152603 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152605 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152608 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152611 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152614 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152617 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152619 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152622 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152625 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152627 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152630 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152633 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152636 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152639 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152642 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152644 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152647 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152650 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152653 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152655 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:17:49.159156 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152658 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152668 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152679 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152683 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152686 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152689 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:49.152692 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.152697 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.154172 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.156955 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.158585 2566 server.go:1019] "Starting client certificate rotation" Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.158689 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:17:49.159809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.158722 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:17:49.161106 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.161090 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:17:49.165258 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.165237 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:17:49.176111 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.176088 2566 log.go:25] "Validated CRI v1 runtime API" Jul 06 18:17:49.179136 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.179115 2566 log.go:25] "Validated CRI v1 image API" Jul 06 18:17:49.180045 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.180031 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 18:17:49.182729 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.182708 2566 fs.go:135] Filesystem UUIDs: map[73df28b4-d3ca-4528-aaa7-b65ef6fc2f4c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d1ba006b-d45a-4d56-84bf-c3a5d04034fc:/dev/nvme0n1p4] Jul 06 18:17:49.182778 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.182731 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 18:17:49.189029 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.188898 2566 manager.go:216] Machine: {Timestamp:2026-07-06 18:17:49.187594006 +0000 UTC m=+0.334134922 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097134 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289f4d38d293e32e8d432515f763ea SystemUUID:ec289f4d-38d2-93e3-2e8d-432515f763ea BootID:f8170f60-327f-46aa-b3a6-868e7618d691 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:44:b2:53:8e:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:44:b2:53:8e:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:2f:35:d0:72:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 18:17:49.189029 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.189026 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 18:17:49.189150 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.189132 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 18:17:49.192109 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.192073 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 18:17:49.192269 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.192110 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-250.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 18:17:49.194075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.194057 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:17:49.194115 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.194107 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 18:17:49.194152 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.194122 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 18:17:49.194152 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.194131 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:17:49.194819 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.194808 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:17:49.196135 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.196123 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 18:17:49.199768 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199754 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:17:49.199902 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199892 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 18:17:49.199944 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199926 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 06 18:17:49.199980 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199958 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 18:17:49.199980 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199970 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 18:17:49.200061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199982 2566 kubelet.go:398] "Adding apiserver pod source" Jul 06 18:17:49.200061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.199992 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 18:17:49.200546 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.200527 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:17:49.200597 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.200558 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:17:49.204211 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.204179 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 18:17:49.206499 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.206485 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 18:17:49.206569 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.206526 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 18:17:49.207764 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207752 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 18:17:49.207805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207770 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 18:17:49.207805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207777 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 18:17:49.207805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207783 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 18:17:49.207805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207791 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 18:17:49.207805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207797 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 18:17:49.207805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207806 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 18:17:49.207972 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207811 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 18:17:49.207972 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207818 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 18:17:49.207972 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207824 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 18:17:49.207972 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207833 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 18:17:49.207972 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.207838 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 18:17:49.208423 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.208412 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 18:17:49.208458 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.208426 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 18:17:49.212231 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.212215 2566 server.go:1268] "Started kubelet" Jul 06 18:17:49.212384 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.212357 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 18:17:49.212492 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.212367 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 18:17:49.212492 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.212460 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 18:17:49.213303 ip-10-0-133-250 systemd[1]: Started Kubernetes Kubelet. Jul 06 18:17:49.213664 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.213559 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 18:17:49.213664 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.213652 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 18:17:49.213981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.213904 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-250.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 18:17:49.214642 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.214623 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 18:17:49.215132 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.215117 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 06 18:17:49.218102 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.218082 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 18:17:49.219750 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.218858 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-250.ec2.internal.18bfc62be05e623e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-250.ec2.internal,UID:ip-10-0-133-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-250.ec2.internal,},FirstTimestamp:2026-07-06 18:17:49.212181054 +0000 UTC m=+0.358721972,LastTimestamp:2026-07-06 18:17:49.212181054 +0000 UTC m=+0.358721972,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-250.ec2.internal,}" Jul 06 18:17:49.220650 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.220627 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 18:17:49.220650 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.220632 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 18:17:49.220819 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.220747 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.221370 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221351 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 18:17:49.221470 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221376 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 18:17:49.221470 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221351 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 18:17:49.221470 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221432 2566 factory.go:55] Registering systemd factory Jul 06 18:17:49.221470 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221448 2566 factory.go:223] Registration of the systemd container factory successfully Jul 06 18:17:49.221844 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221828 2566 factory.go:153] Registering CRI-O factory Jul 06 18:17:49.221844 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221845 2566 factory.go:223] Registration of the crio container factory successfully Jul 06 18:17:49.221965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221897 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 18:17:49.221965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221940 2566 factory.go:103] Registering Raw factory Jul 06 18:17:49.221965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.221958 2566 manager.go:1196] Started watching for new ooms in manager Jul 06 18:17:49.222289 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.222276 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 06 18:17:49.222329 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.222290 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 06 18:17:49.222604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.222588 2566 manager.go:349] Starting recovery of all containers Jul 06 18:17:49.223583 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.223557 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wcns" Jul 06 18:17:49.224998 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.224964 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 18:17:49.225106 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.224997 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 18:17:49.231815 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.231641 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wcns" Jul 06 18:17:49.234293 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.234272 2566 manager.go:354] Recovery completed Jul 06 18:17:49.240460 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.240442 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.242402 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.242373 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.242513 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.242422 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.242513 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.242434 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.243095 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243079 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 18:17:49.243095 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243092 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 18:17:49.243236 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243100 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.243236 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243107 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:17:49.243837 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243822 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.243924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243856 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.243924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.243872 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.244571 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.244557 2566 policy_none.go:49] "None policy: Start" Jul 06 18:17:49.244635 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.244576 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 18:17:49.244635 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.244591 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 18:17:49.245090 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.245015 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-250.ec2.internal.18bfc62be22ba941 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-250.ec2.internal,UID:ip-10-0-133-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-250.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-250.ec2.internal,},FirstTimestamp:2026-07-06 18:17:49.242411329 +0000 UTC m=+0.388952248,LastTimestamp:2026-07-06 18:17:49.242411329 +0000 UTC m=+0.388952248,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-250.ec2.internal,}" Jul 06 18:17:49.245695 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.245681 2566 policy_none.go:47] "Start" Jul 06 18:17:49.296128 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296107 2566 manager.go:337] "Starting Device Plugin manager" Jul 06 18:17:49.296272 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.296153 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 18:17:49.296272 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296169 2566 server.go:87] "Starting device plugin registration server" Jul 06 18:17:49.296498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296484 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 18:17:49.296565 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296501 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 18:17:49.296612 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296585 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 18:17:49.296707 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296691 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 18:17:49.296766 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.296722 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 18:17:49.297360 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.297335 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 18:17:49.297474 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.297416 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.396853 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.396750 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.399595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.399577 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.399672 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.399611 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.399672 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.399622 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.399672 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.399651 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.407452 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.407434 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.407504 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.407461 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-250.ec2.internal\": node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.423407 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.423380 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.521491 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.521448 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 18:17:49.522780 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.522761 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 18:17:49.522924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.522803 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 18:17:49.522924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.522830 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 18:17:49.522924 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.522868 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 18:17:49.523471 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.523451 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.525627 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.525611 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:17:49.626309 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.626241 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal"] Jul 06 18:17:49.626506 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.626262 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.626506 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.626376 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.627492 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.627473 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.627594 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.627508 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.627594 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.627520 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.629266 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.629252 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.629418 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.629402 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.629484 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.629472 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.630090 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.630071 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.630174 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.630106 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.630174 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.630117 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.630174 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.630077 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.630289 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.630180 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.630289 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.630191 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.631460 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.631442 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.631510 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.631481 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:17:49.632267 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.632244 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:17:49.632373 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.632280 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:17:49.632373 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.632297 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:17:49.656152 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.656093 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-250.ec2.internal\" not found" node="ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.660111 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.660094 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-250.ec2.internal\" not found" node="ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.726865 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.726831 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.727024 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.726875 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/212e78a38a7830a46f3ee0f8a68d9ed2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal\" (UID: \"212e78a38a7830a46f3ee0f8a68d9ed2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.727024 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.726908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/212e78a38a7830a46f3ee0f8a68d9ed2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal\" (UID: \"212e78a38a7830a46f3ee0f8a68d9ed2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.727024 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.726936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c5d0c0b8e27dab8be106a1a4a27f027-config\") pod \"kube-apiserver-proxy-ip-10-0-133-250.ec2.internal\" (UID: \"7c5d0c0b8e27dab8be106a1a4a27f027\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.827702 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.827654 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.827871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.827740 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/212e78a38a7830a46f3ee0f8a68d9ed2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal\" (UID: \"212e78a38a7830a46f3ee0f8a68d9ed2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.827871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.827748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/212e78a38a7830a46f3ee0f8a68d9ed2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal\" (UID: \"212e78a38a7830a46f3ee0f8a68d9ed2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.827871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.827810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/212e78a38a7830a46f3ee0f8a68d9ed2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal\" (UID: \"212e78a38a7830a46f3ee0f8a68d9ed2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.827871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.827827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c5d0c0b8e27dab8be106a1a4a27f027-config\") pod \"kube-apiserver-proxy-ip-10-0-133-250.ec2.internal\" (UID: \"7c5d0c0b8e27dab8be106a1a4a27f027\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.827871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.827851 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c5d0c0b8e27dab8be106a1a4a27f027-config\") pod \"kube-apiserver-proxy-ip-10-0-133-250.ec2.internal\" (UID: \"7c5d0c0b8e27dab8be106a1a4a27f027\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.827871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.827868 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/212e78a38a7830a46f3ee0f8a68d9ed2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal\" (UID: \"212e78a38a7830a46f3ee0f8a68d9ed2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.928146 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:49.928073 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:49.962899 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.962854 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:49.964039 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:49.964023 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" Jul 06 18:17:50.029217 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:50.029176 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:50.130308 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:50.130270 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:50.158166 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.158136 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 18:17:50.158739 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.158312 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:17:50.231270 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:50.231191 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:50.236051 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.236017 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:12:50 +0000 UTC" deadline="2028-03-29 20:38:21.744852524 +0000 UTC" Jul 06 18:17:50.236051 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.236050 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15170h20m31.508805337s" Jul 06 18:17:50.332306 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:50.332248 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:50.433436 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:50.433396 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:50.534374 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:50.534289 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-250.ec2.internal\" not found" Jul 06 18:17:50.546471 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.546447 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:17:50.555037 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.555014 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:17:50.570380 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:50.570353 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212e78a38a7830a46f3ee0f8a68d9ed2.slice/crio-ba91afe6ac952b511ae90ff0f26d0f50bfae1e4af6b4a4e65acd412c7d282b8b WatchSource:0}: Error finding container ba91afe6ac952b511ae90ff0f26d0f50bfae1e4af6b4a4e65acd412c7d282b8b: Status 404 returned error can't find the container with id ba91afe6ac952b511ae90ff0f26d0f50bfae1e4af6b4a4e65acd412c7d282b8b Jul 06 18:17:50.570853 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:50.570830 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5d0c0b8e27dab8be106a1a4a27f027.slice/crio-1c1af555f678f629fec45e57cb21296c79cd69ccad8c17dfb18f623218404ba6 WatchSource:0}: Error finding container 1c1af555f678f629fec45e57cb21296c79cd69ccad8c17dfb18f623218404ba6: Status 404 returned error can't find the container with id 1c1af555f678f629fec45e57cb21296c79cd69ccad8c17dfb18f623218404ba6 Jul 06 18:17:50.573614 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.573593 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:17:50.577103 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.577087 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:17:50.622428 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.622389 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" Jul 06 18:17:50.639332 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.639291 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:17:50.640411 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.640383 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" Jul 06 18:17:50.650131 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.650114 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:17:50.994460 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:50.994375 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:17:51.201713 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.201680 2566 apiserver.go:52] "Watching apiserver" Jul 06 18:17:51.215427 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.214829 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 18:17:51.215615 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.215593 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-jfvv2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp","openshift-cluster-node-tuning-operator/tuned-sn9ql","openshift-image-registry/node-ca-ccpbk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal","openshift-multus/network-metrics-daemon-46jsb","openshift-network-operator/iptables-alerter-7n5ch","openshift-ovn-kubernetes/ovnkube-node-d4ljq","kube-system/global-pull-secret-syncer-s6kxp","kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal","openshift-dns/node-resolver-xqglp","openshift-multus/multus-additional-cni-plugins-ps2sp","openshift-multus/multus-jjzz7","openshift-network-diagnostics/network-check-target-4h4tn"] Jul 06 18:17:51.218685 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.217764 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.221274 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.219487 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.221562 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.221433 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.223492 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.223834 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.223913 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cmhhj\"" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.224026 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qp82k\"" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.224217 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.224221 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.224965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.224330 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.225371 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.225306 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.227584 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.227274 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:51.227584 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.227356 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:17:51.228173 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.227987 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 18:17:51.229024 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.228920 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.229024 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.228990 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:17:51.230938 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.230189 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.230938 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.230245 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:17:51.232231 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.232208 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.233053 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.232875 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.233053 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.232932 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mvlrl\"" Jul 06 18:17:51.233261 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.233213 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.233510 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.233489 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.233913 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.233775 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.234007 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.233916 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 18:17:51.234007 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.233983 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x8b2h\"" Jul 06 18:17:51.234237 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.234216 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.234749 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.234722 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.235143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.235123 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 18:17:51.236616 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.235902 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 18:17:51.236616 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.235946 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.236616 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.236132 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.237739 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.237719 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238259 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238554 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238639 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fsp4r\"" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238639 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mtfz8\"" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88jn9\" (UniqueName: \"kubernetes.io/projected/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-kube-api-access-88jn9\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-modprobe-d\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238911 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-host\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/610d0089-73b6-480c-a13a-26b05ed72426-iptables-alerter-script\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238854 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-hosts-file\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.238991 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-kubernetes\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-systemd\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239037 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-var-lib-kubelet\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysconfig\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239141 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysctl-conf\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239156 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-lib-modules\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239194 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/759076dc-9f6a-490d-99df-6722be4917a6-etc-tuned\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/759076dc-9f6a-490d-99df-6722be4917a6-tmp\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239310 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/610d0089-73b6-480c-a13a-26b05ed72426-host-slash\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239357 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pv4k\" (UniqueName: \"kubernetes.io/projected/610d0089-73b6-480c-a13a-26b05ed72426-kube-api-access-2pv4k\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239385 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239398 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysctl-d\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239495 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239526 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vz5jb\"" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-run\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-sys\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5zw8\" (UniqueName: \"kubernetes.io/projected/759076dc-9f6a-490d-99df-6722be4917a6-kube-api-access-r5zw8\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239640 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-tmp-dir\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239779 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239952 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 18:17:51.239985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.239968 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 18:17:51.240807 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.240052 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 18:17:51.240807 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.240241 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 18:17:51.240807 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.240518 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jwdcf\"" Jul 06 18:17:51.243822 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.243032 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 18:17:51.243822 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.243272 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 18:17:51.243822 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.243280 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 18:17:51.243822 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.243601 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tfsbl\"" Jul 06 18:17:51.244073 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.243897 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 18:17:51.250955 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.250898 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:17:51.282058 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.282018 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8j2h" Jul 06 18:17:51.293307 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.293277 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8j2h" Jul 06 18:17:51.323197 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.323170 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 18:17:51.340171 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340126 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-var-lib-kubelet\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.340417 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6t8q\" (UniqueName: \"kubernetes.io/projected/0de316fb-6637-4086-8cb1-e077df4a27b5-kube-api-access-g6t8q\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.340417 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/819d5ca7-3999-461d-8609-5c7c9136ffe2-cni-binary-copy\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.340417 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340314 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-cni-multus\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.340417 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340345 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-var-lib-kubelet\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.340417 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkkt5\" (UniqueName: \"kubernetes.io/projected/f47efaae-fb25-4ea2-96af-494873864f0d-kube-api-access-pkkt5\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysconfig\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340500 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pv4k\" (UniqueName: \"kubernetes.io/projected/610d0089-73b6-480c-a13a-26b05ed72426-kube-api-access-2pv4k\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-socket-dir-parent\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysctl-d\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-tmp-dir\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.340670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340619 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysconfig\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0de316fb-6637-4086-8cb1-e077df4a27b5-host\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrq7s\" (UniqueName: \"kubernetes.io/projected/819d5ca7-3999-461d-8609-5c7c9136ffe2-kube-api-access-wrq7s\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340779 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysctl-d\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvnl2\" (UniqueName: \"kubernetes.io/projected/35b93075-0cf6-419f-a00d-22a4b7f57edc-kube-api-access-hvnl2\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-os-release\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340862 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-tmp-dir\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340875 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.340914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kmxg\" (UniqueName: \"kubernetes.io/projected/d5418dba-da69-4fbd-adb5-735932ed01e5-kube-api-access-9kmxg\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-kubelet\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-daemon-config\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.340976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2p9n\" (UniqueName: \"kubernetes.io/projected/d9c34f33-2566-42dd-976b-d570b857a4e0-kube-api-access-g2p9n\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341001 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b429876b-7071-41ab-985a-a32f0b9ea8f4-kubelet-config\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-socket-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-node-log\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341151 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysctl-conf\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-slash\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-log-socket\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9e29be-1af8-419b-9847-e53db35a1c78-agent-certs\") pod \"konnectivity-agent-jfvv2\" (UID: \"fc9e29be-1af8-419b-9847-e53db35a1c78\") " pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-system-cni-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341371 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-sysctl-conf\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-kubelet\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341426 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-cni-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-hostroot\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-multus-certs\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-cni-bin\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-systemd\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-conf-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341608 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-etc-kubernetes\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341645 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-systemd\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-systemd\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-lib-modules\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.341983 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/759076dc-9f6a-490d-99df-6722be4917a6-etc-tuned\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341821 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-registration-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-sys-fs\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341880 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b429876b-7071-41ab-985a-a32f0b9ea8f4-dbus\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341905 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-systemd-units\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-cnibin\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.341966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-run\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88jn9\" (UniqueName: \"kubernetes.io/projected/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-kube-api-access-88jn9\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-os-release\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-k8s-cni-cncf-io\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342111 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-ovn\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-modprobe-d\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342169 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-host\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/610d0089-73b6-480c-a13a-26b05ed72426-iptables-alerter-script\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-cni-bin\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-etc-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-cni-netd\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.342738 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-system-cni-dir\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342309 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-run\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-kubernetes\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-cnibin\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342392 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-kubernetes\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-env-overrides\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342504 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-etc-modprobe-d\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342572 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-host\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342758 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.342852 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-lib-modules\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343134 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/610d0089-73b6-480c-a13a-26b05ed72426-iptables-alerter-script\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovn-node-metrics-cert\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/759076dc-9f6a-490d-99df-6722be4917a6-tmp\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/610d0089-73b6-480c-a13a-26b05ed72426-host-slash\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343293 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovnkube-config\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0de316fb-6637-4086-8cb1-e077df4a27b5-serviceca\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-sys\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5zw8\" (UniqueName: \"kubernetes.io/projected/759076dc-9f6a-490d-99df-6722be4917a6-kube-api-access-r5zw8\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.343604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343437 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-device-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-run-netns\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-var-lib-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343521 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/759076dc-9f6a-490d-99df-6722be4917a6-sys\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343538 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343575 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-hosts-file\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343593 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/610d0089-73b6-480c-a13a-26b05ed72426-host-slash\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-netns\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovnkube-script-lib\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9e29be-1af8-419b-9847-e53db35a1c78-konnectivity-ca\") pod \"konnectivity-agent-jfvv2\" (UID: \"fc9e29be-1af8-419b-9847-e53db35a1c78\") " pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.344580 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.343666 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-hosts-file\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.347194 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.347167 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/759076dc-9f6a-490d-99df-6722be4917a6-tmp\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.348687 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.348664 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/759076dc-9f6a-490d-99df-6722be4917a6-etc-tuned\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.352743 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.352696 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pv4k\" (UniqueName: \"kubernetes.io/projected/610d0089-73b6-480c-a13a-26b05ed72426-kube-api-access-2pv4k\") pod \"iptables-alerter-7n5ch\" (UID: \"610d0089-73b6-480c-a13a-26b05ed72426\") " pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.353957 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.353801 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5zw8\" (UniqueName: \"kubernetes.io/projected/759076dc-9f6a-490d-99df-6722be4917a6-kube-api-access-r5zw8\") pod \"tuned-sn9ql\" (UID: \"759076dc-9f6a-490d-99df-6722be4917a6\") " pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.356806 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.356782 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88jn9\" (UniqueName: \"kubernetes.io/projected/8d9b7540-f14d-4f63-b8af-3afda5aad1c4-kube-api-access-88jn9\") pod \"node-resolver-xqglp\" (UID: \"8d9b7540-f14d-4f63-b8af-3afda5aad1c4\") " pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.444928 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.444889 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.445124 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.444979 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkkt5\" (UniqueName: \"kubernetes.io/projected/f47efaae-fb25-4ea2-96af-494873864f0d-kube-api-access-pkkt5\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.445124 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-socket-dir-parent\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.445124 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.445124 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0de316fb-6637-4086-8cb1-e077df4a27b5-host\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.445124 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445095 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrq7s\" (UniqueName: \"kubernetes.io/projected/819d5ca7-3999-461d-8609-5c7c9136ffe2-kube-api-access-wrq7s\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.445398 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445139 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-socket-dir-parent\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.445546 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445514 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0de316fb-6637-4086-8cb1-e077df4a27b5-host\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.445682 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445656 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.445742 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvnl2\" (UniqueName: \"kubernetes.io/projected/35b93075-0cf6-419f-a00d-22a4b7f57edc-kube-api-access-hvnl2\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.445742 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445727 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-os-release\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.445845 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.445845 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445779 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.445845 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kmxg\" (UniqueName: \"kubernetes.io/projected/d5418dba-da69-4fbd-adb5-735932ed01e5-kube-api-access-9kmxg\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.445845 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.445789 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:51.445845 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-kubelet\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.445845 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-daemon-config\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445866 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2p9n\" (UniqueName: \"kubernetes.io/projected/d9c34f33-2566-42dd-976b-d570b857a4e0-kube-api-access-g2p9n\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.445921 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:51.945868381 +0000 UTC m=+3.092409300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445968 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-kubelet\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.445977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446003 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-os-release\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446068 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b429876b-7071-41ab-985a-a32f0b9ea8f4-kubelet-config\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.446137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446102 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.446499 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446296 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b429876b-7071-41ab-985a-a32f0b9ea8f4-kubelet-config\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.446630 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446603 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-socket-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.446688 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-socket-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.446744 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.446744 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446726 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-node-log\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.446846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.446846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-slash\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.446846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446813 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-log-socket\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.446846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446818 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.446846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9e29be-1af8-419b-9847-e53db35a1c78-agent-certs\") pod \"konnectivity-agent-jfvv2\" (UID: \"fc9e29be-1af8-419b-9847-e53db35a1c78\") " pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.447091 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446857 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447091 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446867 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-daemon-config\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447091 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446875 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.447091 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-system-cni-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447091 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447003 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-kubelet\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447501 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447291 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-log-socket\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447553 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446871 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-node-log\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447553 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-kubelet\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447648 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446882 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-slash\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447648 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.447648 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-cni-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447648 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.446933 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-system-cni-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447692 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-cni-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-hostroot\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-multus-certs\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447765 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-hostroot\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447791 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-cni-bin\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447820 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-multus-certs\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.447847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-conf-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447865 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-etc-kubernetes\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447873 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-cni-bin\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447927 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-etc-kubernetes\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.447936 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-multus-conf-dir\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.448102 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.448201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.448126 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-systemd\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.448576 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.448224 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f47efaae-fb25-4ea2-96af-494873864f0d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.450961 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.450934 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9e29be-1af8-419b-9847-e53db35a1c78-agent-certs\") pod \"konnectivity-agent-jfvv2\" (UID: \"fc9e29be-1af8-419b-9847-e53db35a1c78\") " pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.451617 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-systemd\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.451753 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.451753 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-registration-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.451753 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-sys-fs\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451756 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b429876b-7071-41ab-985a-a32f0b9ea8f4-dbus\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451809 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-systemd-units\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451816 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451835 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-registration-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451842 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-cnibin\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451895 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-cnibin\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-os-release\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.451962 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451909 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-sys-fs\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.452324 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.451962 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-os-release\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.452324 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452033 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-k8s-cni-cncf-io\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.452324 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452051 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-k8s-cni-cncf-io\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.452556 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452042 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-systemd-units\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.452556 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452520 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b429876b-7071-41ab-985a-a32f0b9ea8f4-dbus\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-ovn\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-cni-bin\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-etc-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-cni-netd\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-system-cni-dir\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452694 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-cni-bin\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452705 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-cnibin\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452715 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-etc-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.452728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452680 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-run-ovn\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-env-overrides\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovn-node-metrics-cert\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452784 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-cnibin\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-cni-netd\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452809 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovnkube-config\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0de316fb-6637-4086-8cb1-e077df4a27b5-serviceca\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-device-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452914 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9c34f33-2566-42dd-976b-d570b857a4e0-device-dir\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-run-netns\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-var-lib-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.452999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-netns\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovnkube-script-lib\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9e29be-1af8-419b-9847-e53db35a1c78-konnectivity-ca\") pod \"konnectivity-agent-jfvv2\" (UID: \"fc9e29be-1af8-419b-9847-e53db35a1c78\") " pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.453123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453130 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6t8q\" (UniqueName: \"kubernetes.io/projected/0de316fb-6637-4086-8cb1-e077df4a27b5-kube-api-access-g6t8q\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453146 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/819d5ca7-3999-461d-8609-5c7c9136ffe2-cni-binary-copy\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453163 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-cni-multus\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-var-lib-cni-multus\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453243 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-run-netns\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453249 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0de316fb-6637-4086-8cb1-e077df4a27b5-serviceca\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453258 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-env-overrides\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453266 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-var-lib-openvswitch\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453304 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f47efaae-fb25-4ea2-96af-494873864f0d-system-cni-dir\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/819d5ca7-3999-461d-8609-5c7c9136ffe2-host-run-netns\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.453637 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453656 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovnkube-config\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.453696 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:51.953675842 +0000 UTC m=+3.100216765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453702 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35b93075-0cf6-419f-a00d-22a4b7f57edc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.453986 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.453892 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/819d5ca7-3999-461d-8609-5c7c9136ffe2-cni-binary-copy\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.454671 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.454015 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovnkube-script-lib\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.454671 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.454137 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9e29be-1af8-419b-9847-e53db35a1c78-konnectivity-ca\") pod \"konnectivity-agent-jfvv2\" (UID: \"fc9e29be-1af8-419b-9847-e53db35a1c78\") " pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.455840 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.455814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35b93075-0cf6-419f-a00d-22a4b7f57edc-ovn-node-metrics-cert\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.456486 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.456459 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkkt5\" (UniqueName: \"kubernetes.io/projected/f47efaae-fb25-4ea2-96af-494873864f0d-kube-api-access-pkkt5\") pod \"multus-additional-cni-plugins-ps2sp\" (UID: \"f47efaae-fb25-4ea2-96af-494873864f0d\") " pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.456751 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.456735 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:17:51.456840 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.456773 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:17:51.456840 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.456784 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:51.456840 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.456840 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:51.956820688 +0000 UTC m=+3.103361605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:51.457312 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.457292 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrq7s\" (UniqueName: \"kubernetes.io/projected/819d5ca7-3999-461d-8609-5c7c9136ffe2-kube-api-access-wrq7s\") pod \"multus-jjzz7\" (UID: \"819d5ca7-3999-461d-8609-5c7c9136ffe2\") " pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.459560 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.459536 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2p9n\" (UniqueName: \"kubernetes.io/projected/d9c34f33-2566-42dd-976b-d570b857a4e0-kube-api-access-g2p9n\") pod \"aws-ebs-csi-driver-node-v98kp\" (UID: \"d9c34f33-2566-42dd-976b-d570b857a4e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.459820 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.459798 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kmxg\" (UniqueName: \"kubernetes.io/projected/d5418dba-da69-4fbd-adb5-735932ed01e5-kube-api-access-9kmxg\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.460080 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.460061 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvnl2\" (UniqueName: \"kubernetes.io/projected/35b93075-0cf6-419f-a00d-22a4b7f57edc-kube-api-access-hvnl2\") pod \"ovnkube-node-d4ljq\" (UID: \"35b93075-0cf6-419f-a00d-22a4b7f57edc\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.464463 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.464442 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6t8q\" (UniqueName: \"kubernetes.io/projected/0de316fb-6637-4086-8cb1-e077df4a27b5-kube-api-access-g6t8q\") pod \"node-ca-ccpbk\" (UID: \"0de316fb-6637-4086-8cb1-e077df4a27b5\") " pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.528568 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.528453 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" event={"ID":"212e78a38a7830a46f3ee0f8a68d9ed2","Type":"ContainerStarted","Data":"ba91afe6ac952b511ae90ff0f26d0f50bfae1e4af6b4a4e65acd412c7d282b8b"} Jul 06 18:17:51.529782 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.529746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" event={"ID":"7c5d0c0b8e27dab8be106a1a4a27f027","Type":"ContainerStarted","Data":"1c1af555f678f629fec45e57cb21296c79cd69ccad8c17dfb18f623218404ba6"} Jul 06 18:17:51.536058 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.536031 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7n5ch" Jul 06 18:17:51.545525 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.545502 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xqglp" Jul 06 18:17:51.547461 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:51.547429 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610d0089_73b6_480c_a13a_26b05ed72426.slice/crio-92765b74e1abaad79cb0951dd2a50da3649fe7bf3c5bf0e809677b2c96ddd1e6 WatchSource:0}: Error finding container 92765b74e1abaad79cb0951dd2a50da3649fe7bf3c5bf0e809677b2c96ddd1e6: Status 404 returned error can't find the container with id 92765b74e1abaad79cb0951dd2a50da3649fe7bf3c5bf0e809677b2c96ddd1e6 Jul 06 18:17:51.558702 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:51.558672 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9b7540_f14d_4f63_b8af_3afda5aad1c4.slice/crio-02aaf27d666c4c46f44c766baf3f97092679abb2b0f6c5a153179a6ef6da6a92 WatchSource:0}: Error finding container 02aaf27d666c4c46f44c766baf3f97092679abb2b0f6c5a153179a6ef6da6a92: Status 404 returned error can't find the container with id 02aaf27d666c4c46f44c766baf3f97092679abb2b0f6c5a153179a6ef6da6a92 Jul 06 18:17:51.560264 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.560233 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" Jul 06 18:17:51.568487 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.568428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ccpbk" Jul 06 18:17:51.569887 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:51.569859 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759076dc_9f6a_490d_99df_6722be4917a6.slice/crio-b5c40d6158b30802dde8b381a9dda19f7884f79e520060cc44355847493f5052 WatchSource:0}: Error finding container b5c40d6158b30802dde8b381a9dda19f7884f79e520060cc44355847493f5052: Status 404 returned error can't find the container with id b5c40d6158b30802dde8b381a9dda19f7884f79e520060cc44355847493f5052 Jul 06 18:17:51.577039 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.577015 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jjzz7" Jul 06 18:17:51.581571 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:51.581544 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de316fb_6637_4086_8cb1_e077df4a27b5.slice/crio-85f64387294c2fd3bd8e9bfa3f182d0e3c1455e48683ba419d203e3b411d6be5 WatchSource:0}: Error finding container 85f64387294c2fd3bd8e9bfa3f182d0e3c1455e48683ba419d203e3b411d6be5: Status 404 returned error can't find the container with id 85f64387294c2fd3bd8e9bfa3f182d0e3c1455e48683ba419d203e3b411d6be5 Jul 06 18:17:51.585880 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.585854 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" Jul 06 18:17:51.586936 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:51.586911 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819d5ca7_3999_461d_8609_5c7c9136ffe2.slice/crio-d3e6d76e7a94125a2cbb4c3f5eaa9e6c94de84004747f557211d0c86f78feba0 WatchSource:0}: Error finding container d3e6d76e7a94125a2cbb4c3f5eaa9e6c94de84004747f557211d0c86f78feba0: Status 404 returned error can't find the container with id d3e6d76e7a94125a2cbb4c3f5eaa9e6c94de84004747f557211d0c86f78feba0 Jul 06 18:17:51.595316 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.595292 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:17:51.603459 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.603428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:17:51.610454 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.610430 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" Jul 06 18:17:51.957594 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.957496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:51.957594 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.957552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:51.957594 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:51.957588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957670 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957699 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957727 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957753 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957770 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957756 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:52.957733783 +0000 UTC m=+4.104274689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957839 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:52.957820149 +0000 UTC m=+4.104361059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:51.957882 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:51.957863 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:52.957853228 +0000 UTC m=+4.104394136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:52.296847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.296671 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:12:52 +0000 UTC" deadline="2028-02-27 11:38:15.651478089 +0000 UTC" Jul 06 18:17:52.296847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.296710 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14417h20m23.354773052s" Jul 06 18:17:52.524072 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.524030 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:52.524253 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.524187 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:17:52.524714 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.524690 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:52.524832 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.524810 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:17:52.533299 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.533268 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7n5ch" event={"ID":"610d0089-73b6-480c-a13a-26b05ed72426","Type":"ContainerStarted","Data":"92765b74e1abaad79cb0951dd2a50da3649fe7bf3c5bf0e809677b2c96ddd1e6"} Jul 06 18:17:52.534949 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.534921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jjzz7" event={"ID":"819d5ca7-3999-461d-8609-5c7c9136ffe2","Type":"ContainerStarted","Data":"d3e6d76e7a94125a2cbb4c3f5eaa9e6c94de84004747f557211d0c86f78feba0"} Jul 06 18:17:52.536418 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.536382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ccpbk" event={"ID":"0de316fb-6637-4086-8cb1-e077df4a27b5","Type":"ContainerStarted","Data":"85f64387294c2fd3bd8e9bfa3f182d0e3c1455e48683ba419d203e3b411d6be5"} Jul 06 18:17:52.539319 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.539294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" event={"ID":"759076dc-9f6a-490d-99df-6722be4917a6","Type":"ContainerStarted","Data":"b5c40d6158b30802dde8b381a9dda19f7884f79e520060cc44355847493f5052"} Jul 06 18:17:52.540772 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.540746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xqglp" event={"ID":"8d9b7540-f14d-4f63-b8af-3afda5aad1c4","Type":"ContainerStarted","Data":"02aaf27d666c4c46f44c766baf3f97092679abb2b0f6c5a153179a6ef6da6a92"} Jul 06 18:17:52.968846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.968770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:52.968846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.968838 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:52.969098 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:52.968876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:52.969098 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969052 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:17:52.969098 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969074 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:17:52.969098 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969088 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:52.969317 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969155 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:54.969134988 +0000 UTC m=+6.115675919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:52.969761 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969598 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:52.969761 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969657 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:54.969640338 +0000 UTC m=+6.116181242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:52.969761 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969729 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:52.969761 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:52.969764 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:54.96975348 +0000 UTC m=+6.116294385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:53.236650 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:53.236615 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b93075_0cf6_419f_a00d_22a4b7f57edc.slice/crio-bda8a4ad0830e64c4cf48f8b574018bf484f219c9b684286496276f019f55824 WatchSource:0}: Error finding container bda8a4ad0830e64c4cf48f8b574018bf484f219c9b684286496276f019f55824: Status 404 returned error can't find the container with id bda8a4ad0830e64c4cf48f8b574018bf484f219c9b684286496276f019f55824 Jul 06 18:17:53.245781 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:53.245542 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9e29be_1af8_419b_9847_e53db35a1c78.slice/crio-ca89d3861c90a3f4fc0ebf0caf58bda402f2faad3051eadbd4e964248d28aabf WatchSource:0}: Error finding container ca89d3861c90a3f4fc0ebf0caf58bda402f2faad3051eadbd4e964248d28aabf: Status 404 returned error can't find the container with id ca89d3861c90a3f4fc0ebf0caf58bda402f2faad3051eadbd4e964248d28aabf Jul 06 18:17:53.260937 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:17:53.260902 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47efaae_fb25_4ea2_96af_494873864f0d.slice/crio-f4dc749bb79a3e98749dc3cb6183e9f6cd9147955ef296ffbaddb1ffd40a1b6b WatchSource:0}: Error finding container f4dc749bb79a3e98749dc3cb6183e9f6cd9147955ef296ffbaddb1ffd40a1b6b: Status 404 returned error can't find the container with id f4dc749bb79a3e98749dc3cb6183e9f6cd9147955ef296ffbaddb1ffd40a1b6b Jul 06 18:17:53.297809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.297746 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:12:52 +0000 UTC" deadline="2027-12-07 15:07:50.010252446 +0000 UTC" Jul 06 18:17:53.297809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.297808 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12452h49m56.712449507s" Jul 06 18:17:53.524422 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.523787 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:53.524422 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:53.523934 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:17:53.544912 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.544870 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jfvv2" event={"ID":"fc9e29be-1af8-419b-9847-e53db35a1c78","Type":"ContainerStarted","Data":"ca89d3861c90a3f4fc0ebf0caf58bda402f2faad3051eadbd4e964248d28aabf"} Jul 06 18:17:53.548622 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.548585 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"bda8a4ad0830e64c4cf48f8b574018bf484f219c9b684286496276f019f55824"} Jul 06 18:17:53.552320 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.552282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" event={"ID":"d9c34f33-2566-42dd-976b-d570b857a4e0","Type":"ContainerStarted","Data":"bf4e8d569b2ec046113baf4ef3fa882ec73edcd6162af4c15945f029ec449529"} Jul 06 18:17:53.557872 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.557840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" event={"ID":"7c5d0c0b8e27dab8be106a1a4a27f027","Type":"ContainerStarted","Data":"7fad0f2ea07fb886466900b9b7568e09c66ee1c4468beae55b49a8d9a55cacbe"} Jul 06 18:17:53.561078 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.561008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerStarted","Data":"f4dc749bb79a3e98749dc3cb6183e9f6cd9147955ef296ffbaddb1ffd40a1b6b"} Jul 06 18:17:53.576343 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:53.576260 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-250.ec2.internal" podStartSLOduration=2.576238478 podStartE2EDuration="2.576238478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:17:53.575493495 +0000 UTC m=+4.722034422" watchObservedRunningTime="2026-07-06 18:17:53.576238478 +0000 UTC m=+4.722779404" Jul 06 18:17:54.524041 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.523995 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:54.524573 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.524051 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:54.524573 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.524150 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:17:54.524573 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.524316 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:17:54.566410 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.566330 2566 generic.go:356] "Generic (PLEG): container finished" podID="212e78a38a7830a46f3ee0f8a68d9ed2" containerID="8db056c551a1a099cb6f1641f3068e8f5f7832d8e5579a5e5d2b36f484bfc5ce" exitCode=0 Jul 06 18:17:54.566945 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.566918 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" event={"ID":"212e78a38a7830a46f3ee0f8a68d9ed2","Type":"ContainerDied","Data":"8db056c551a1a099cb6f1641f3068e8f5f7832d8e5579a5e5d2b36f484bfc5ce"} Jul 06 18:17:54.988191 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.988146 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:54.988428 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.988210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:54.988428 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:54.988248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:54.988561 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.988432 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:17:54.988561 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.988452 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:17:54.988561 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.988466 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:54.988561 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.988531 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:58.988510227 +0000 UTC m=+10.135051136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:54.988967 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.988947 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:54.989025 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.989006 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:58.988990173 +0000 UTC m=+10.135531079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:54.989095 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.989068 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:54.989149 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:54.989104 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:58.989092099 +0000 UTC m=+10.135633006 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:55.523774 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:55.523736 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:55.523974 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:55.523891 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:17:56.523941 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:56.523845 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:56.524470 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:56.523998 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:17:56.524554 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:56.524481 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:56.524640 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:56.524605 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:17:56.580325 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:56.580274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xqglp" event={"ID":"8d9b7540-f14d-4f63-b8af-3afda5aad1c4","Type":"ContainerStarted","Data":"8b36b8afbcfba263c9be5b5a8e765b9f7648970054681a7ceb3c95c1a09b4b85"} Jul 06 18:17:56.585228 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:56.585180 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" event={"ID":"212e78a38a7830a46f3ee0f8a68d9ed2","Type":"ContainerStarted","Data":"c96b485f1b9d6a73192b62c0a7c51e4476472d2431979f05f2eed124b4b3dc95"} Jul 06 18:17:56.602897 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:56.602836 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xqglp" podStartSLOduration=2.072284996 podStartE2EDuration="6.60281579s" totalImagesPullingTime="4.530530794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:17:56.602126652 +0000 UTC m=+7.748667578" watchObservedRunningTime="2026-07-06 18:17:56.60281579 +0000 UTC m=+7.749356713" Jul 06 18:17:57.524616 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:57.524579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:57.525061 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:57.524755 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:17:58.523546 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:58.523505 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:58.523743 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:58.523514 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:58.523743 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:58.523658 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:17:58.523876 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:58.523779 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:17:58.592441 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:58.592397 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ccpbk" event={"ID":"0de316fb-6637-4086-8cb1-e077df4a27b5","Type":"ContainerStarted","Data":"452c68b74554624c1cc5dec773dc0ac28f5290c541af093e8ef39b91b8a30fb0"} Jul 06 18:17:58.611056 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:58.610993 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ccpbk" podStartSLOduration=4.10484027 podStartE2EDuration="8.610974624s" totalImagesPullingTime="4.506134354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:17:58.61028783 +0000 UTC m=+9.756828758" watchObservedRunningTime="2026-07-06 18:17:58.610974624 +0000 UTC m=+9.757515550" Jul 06 18:17:58.611252 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:58.611091 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-250.ec2.internal" podStartSLOduration=7.611084734 podStartE2EDuration="7.611084734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:17:56.619535273 +0000 UTC m=+7.766076200" watchObservedRunningTime="2026-07-06 18:17:58.611084734 +0000 UTC m=+9.757625659" Jul 06 18:17:59.026103 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:59.026061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:17:59.026311 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:59.026127 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:59.026311 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:59.026170 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:17:59.026466 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026356 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:59.026466 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026375 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:17:59.026466 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026398 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:17:59.026466 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026412 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:59.026466 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026439 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:07.0264143 +0000 UTC m=+18.172955226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:17:59.026466 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026467 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:07.026449139 +0000 UTC m=+18.172990066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:17:59.026820 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026536 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:59.026820 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.026574 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:07.026560517 +0000 UTC m=+18.173101422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:17:59.525608 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:59.525510 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:17:59.525787 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:17:59.525671 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:17:59.597484 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:59.597439 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7n5ch" event={"ID":"610d0089-73b6-480c-a13a-26b05ed72426","Type":"ContainerStarted","Data":"4bc2aef2aa45e8ed797fc951b99f7f12719d67ec6a63d13c52a08998c4fe0329"} Jul 06 18:17:59.615193 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:17:59.615042 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7n5ch" podStartSLOduration=5.068002527 podStartE2EDuration="9.615021837s" totalImagesPullingTime="4.54701931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:17:59.614706995 +0000 UTC m=+10.761247922" watchObservedRunningTime="2026-07-06 18:17:59.615021837 +0000 UTC m=+10.761562764" Jul 06 18:18:00.523498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:00.523305 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:00.523498 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:00.523464 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:00.523778 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:00.523516 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:00.523778 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:00.523663 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:01.523495 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:01.523292 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:01.523934 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:01.523609 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:02.524069 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:02.524027 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:02.524505 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:02.524076 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:02.524505 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:02.524168 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:02.524505 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:02.524337 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:03.523735 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:03.523701 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:03.523933 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:03.523819 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:04.523641 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:04.523600 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:04.524093 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:04.523605 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:04.524093 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:04.523737 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:04.524093 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:04.523825 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:05.523670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:05.523632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:05.524092 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:05.523759 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:06.523434 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:06.523398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:06.523641 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:06.523515 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:06.523641 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:06.523575 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:06.524035 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:06.523692 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:07.082561 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:07.082526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:07.082579 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:07.082615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082715 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082767 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.082760161 +0000 UTC m=+34.229301078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082826 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.082808048 +0000 UTC m=+34.229348971 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:18:07.082834 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082827 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:18:07.083148 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082848 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:18:07.083148 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082858 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:18:07.083148 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.082908 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.08289432 +0000 UTC m=+34.229435236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:18:07.523609 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:07.523522 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:07.523769 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:07.523688 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:08.523255 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:08.523215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:08.523455 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:08.523357 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:08.523455 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:08.523411 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:08.523556 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:08.523516 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:09.524127 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:09.524097 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:09.524710 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:09.524185 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:10.523589 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:10.523553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:10.523771 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:10.523553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:10.523771 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:10.523671 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:10.523873 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:10.523773 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:11.523656 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.523488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:11.524126 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:11.523743 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:11.621106 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.621071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" event={"ID":"759076dc-9f6a-490d-99df-6722be4917a6","Type":"ContainerStarted","Data":"2e12fc41f6e6fdea3c836aa94cc40d04234cf19b9118a4c3d0d0e4d318581e32"} Jul 06 18:18:11.622536 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.622480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jfvv2" event={"ID":"fc9e29be-1af8-419b-9847-e53db35a1c78","Type":"ContainerStarted","Data":"62eaa722ba84be0ca7d2ae3a4cc65770a0536bc94adb93b692eb16664fd02155"} Jul 06 18:18:11.625001 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.624973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"5ef65a57defa540bb9ba475200322fbc535eb015d3ae23c04a883733d2d1f224"} Jul 06 18:18:11.625101 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.625038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"745c55f0a77e5a6bbee61184b7334ff4b83400f8314f1e79f5a7d3dd2fa5b366"} Jul 06 18:18:11.625101 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.625076 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"891b3bdfed99459cc88b4462ad095f38c93d637f15bd571ecd6ba189ee496b60"} Jul 06 18:18:11.625183 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.625114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"1e742acdd00aa9f99699ed2813c84bdd0e884adea41b53b5b9624a3f21a1c140"} Jul 06 18:18:11.626377 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.626354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" event={"ID":"d9c34f33-2566-42dd-976b-d570b857a4e0","Type":"ContainerStarted","Data":"c81ab2660d3d7c93205c067f9735a8eb05a4a5b46435344d81faa1e3eef54b0a"} Jul 06 18:18:11.627515 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.627496 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jjzz7" event={"ID":"819d5ca7-3999-461d-8609-5c7c9136ffe2","Type":"ContainerStarted","Data":"a380baa33cfe9548a0e5bce7906afb85814c2d82c9b1ac4908b2e184407c025a"} Jul 06 18:18:11.628734 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.628715 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47efaae-fb25-4ea2-96af-494873864f0d" containerID="78fe56abad90a6c500c0a257e7768e244e575908967d9ff47a81cb0c6cfa6b6e" exitCode=0 Jul 06 18:18:11.628792 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.628741 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerDied","Data":"78fe56abad90a6c500c0a257e7768e244e575908967d9ff47a81cb0c6cfa6b6e"} Jul 06 18:18:11.639814 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.639761 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sn9ql" podStartSLOduration=2.149874871 podStartE2EDuration="21.639749384s" totalImagesPullingTime="19.489874513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:11.639284766 +0000 UTC m=+22.785825710" watchObservedRunningTime="2026-07-06 18:18:11.639749384 +0000 UTC m=+22.786290306" Jul 06 18:18:11.655150 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.655103 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jfvv2" podStartSLOduration=3.841660463 podStartE2EDuration="21.655090725s" totalImagesPullingTime="17.813430262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:11.65496635 +0000 UTC m=+22.801507272" watchObservedRunningTime="2026-07-06 18:18:11.655090725 +0000 UTC m=+22.801631647" Jul 06 18:18:11.698759 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:11.698717 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jjzz7" podStartSLOduration=2.113066526 podStartE2EDuration="21.698701332s" totalImagesPullingTime="19.585634806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:11.698161569 +0000 UTC m=+22.844702491" watchObservedRunningTime="2026-07-06 18:18:11.698701332 +0000 UTC m=+22.845242253" Jul 06 18:18:12.523306 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:12.523128 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:12.523472 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:12.523408 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:12.523472 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:12.523174 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:12.523609 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:12.523589 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:12.564537 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:12.564505 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 18:18:12.633082 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:12.633046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"a38b34904917de3beaabcec2830c6c4b98f9b90bfd3a768018611c6beab9e4b0"} Jul 06 18:18:12.633255 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:12.633126 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"3ffafcb7dce3bbb4650d3524c1039682a72ff3e5a486e4e0a17a0b682bf5e4fa"} Jul 06 18:18:12.634619 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:12.634592 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" event={"ID":"d9c34f33-2566-42dd-976b-d570b857a4e0","Type":"ContainerStarted","Data":"fa013e3b48de4f2f08e73daba5879f02d8b5cfbe2412d5d5867d4f5af393f38c"} Jul 06 18:18:13.307897 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:13.307760 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:18:12.56453169Z","UUID":"faf78f94-4c5d-4f65-979b-b2fb8499919c","Handler":null,"Name":"","Endpoint":""} Jul 06 18:18:13.309573 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:13.309549 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 18:18:13.309716 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:13.309597 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 18:18:13.523695 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:13.523666 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:13.523878 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:13.523799 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:14.523459 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:14.523420 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:14.523459 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:14.523449 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:14.524024 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:14.523552 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:14.524024 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:14.523679 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:14.641493 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:14.641460 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"08d11edc1b459b7cef1856ceb9ed43838a20af429c010118ea5507683fdbccad"} Jul 06 18:18:14.643640 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:14.643614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" event={"ID":"d9c34f33-2566-42dd-976b-d570b857a4e0","Type":"ContainerStarted","Data":"5ac4d3225243640eff24a417d7f6d619ac6998e5af9f9d64bd54e6de40b47600"} Jul 06 18:18:14.661497 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:14.661450 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v98kp" podStartSLOduration=4.541011331 podStartE2EDuration="24.661431813s" totalImagesPullingTime="20.120420482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:14.661336261 +0000 UTC m=+25.807877183" watchObservedRunningTime="2026-07-06 18:18:14.661431813 +0000 UTC m=+25.807972737" Jul 06 18:18:15.523434 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:15.523403 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:15.523618 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:15.523510 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:15.956379 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:15.956342 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:18:15.957057 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:15.957034 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:18:16.524032 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.523862 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:16.524610 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:16.524126 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:16.524610 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.523907 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:16.524610 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:16.524212 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:16.652397 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.652354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" event={"ID":"35b93075-0cf6-419f-a00d-22a4b7f57edc","Type":"ContainerStarted","Data":"a634ae4f284861af05591db1e1891aa409001f15043a36ffd3f8c683fd802f6e"} Jul 06 18:18:16.652634 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.652613 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:18:16.652714 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.652690 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:18:16.654399 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.654364 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerStarted","Data":"c6d166d7340fa5eeee895906f92f0fd457c6dca3197608a5a0208db46b41eb12"} Jul 06 18:18:16.654642 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.654617 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:18:16.655044 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.655027 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jfvv2" Jul 06 18:18:16.668939 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.668294 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:18:16.684025 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:16.683985 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" podStartSLOduration=8.797252811 podStartE2EDuration="26.683968653s" totalImagesPullingTime="17.886715842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:16.682755082 +0000 UTC m=+27.829296040" watchObservedRunningTime="2026-07-06 18:18:16.683968653 +0000 UTC m=+27.830509575" Jul 06 18:18:17.523809 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:17.523765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:17.524102 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:17.523916 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:17.657924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:17.657879 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47efaae-fb25-4ea2-96af-494873864f0d" containerID="c6d166d7340fa5eeee895906f92f0fd457c6dca3197608a5a0208db46b41eb12" exitCode=0 Jul 06 18:18:17.658139 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:17.657970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerDied","Data":"c6d166d7340fa5eeee895906f92f0fd457c6dca3197608a5a0208db46b41eb12"} Jul 06 18:18:17.658391 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:17.658371 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:18:17.673960 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:17.673937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:18:18.248906 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.248866 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s6kxp"] Jul 06 18:18:18.249085 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.248977 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:18.249085 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:18.249076 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:18.249546 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.249528 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4h4tn"] Jul 06 18:18:18.249629 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.249618 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:18.249709 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:18.249693 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:18.251290 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.251257 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46jsb"] Jul 06 18:18:18.251405 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.251372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:18.251497 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:18.251478 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:18.662334 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:18.662295 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerStarted","Data":"3fcbad1cdda06681dbaabda0c9126278b59525b72c82ea2c2ada48bcca9fae37"} Jul 06 18:18:19.523876 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:19.523843 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:19.524064 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:19.523939 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:19.524064 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:19.523991 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:19.524187 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:19.524073 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:19.665228 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:19.665021 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47efaae-fb25-4ea2-96af-494873864f0d" containerID="3fcbad1cdda06681dbaabda0c9126278b59525b72c82ea2c2ada48bcca9fae37" exitCode=0 Jul 06 18:18:19.665689 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:19.665112 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerDied","Data":"3fcbad1cdda06681dbaabda0c9126278b59525b72c82ea2c2ada48bcca9fae37"} Jul 06 18:18:20.523181 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:20.523150 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:20.523398 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:20.523257 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:20.670637 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:20.670606 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47efaae-fb25-4ea2-96af-494873864f0d" containerID="0d10c89a73483b8e151894688a30e5824ad8a1daee20a88591ae0149b38c9d79" exitCode=0 Jul 06 18:18:20.671069 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:20.670648 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerDied","Data":"0d10c89a73483b8e151894688a30e5824ad8a1daee20a88591ae0149b38c9d79"} Jul 06 18:18:21.311351 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:21.311304 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xqglp_8d9b7540-f14d-4f63-b8af-3afda5aad1c4/dns-node-resolver/0.log" Jul 06 18:18:21.523732 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:21.523701 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:21.523898 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:21.523807 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:21.523898 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:21.523863 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:21.523978 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:21.523960 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:21.891620 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:21.891590 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ccpbk_0de316fb-6637-4086-8cb1-e077df4a27b5/node-ca/0.log" Jul 06 18:18:22.523821 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:22.523792 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:22.523985 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:22.523886 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:23.101750 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:23.101695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:23.101810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:23.101843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.101890 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.101957 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.101973 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.101979 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs podName:d5418dba-da69-4fbd-adb5-735932ed01e5 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:55.101957331 +0000 UTC m=+66.248498245 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs") pod "network-metrics-daemon-46jsb" (UID: "d5418dba-da69-4fbd-adb5-735932ed01e5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.101989 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.102002 2566 projected.go:196] Error preparing data for projected volume kube-api-access-dchm4 for pod openshift-network-diagnostics/network-check-target-4h4tn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.102011 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret podName:b429876b-7071-41ab-985a-a32f0b9ea8f4 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:55.101997143 +0000 UTC m=+66.248538059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret") pod "global-pull-secret-syncer-s6kxp" (UID: "b429876b-7071-41ab-985a-a32f0b9ea8f4") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:18:23.102241 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.102043 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4 podName:3e58e685-1870-4dde-9709-34eb2675a948 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:55.102030226 +0000 UTC m=+66.248571129 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dchm4" (UniqueName: "kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4") pod "network-check-target-4h4tn" (UID: "3e58e685-1870-4dde-9709-34eb2675a948") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:18:23.523274 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:23.523235 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:23.523579 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.523373 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:23.523579 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:23.523424 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:23.523579 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:23.523510 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:24.524049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:24.524012 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:24.524522 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:24.524121 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:25.523729 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:25.523693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:25.523914 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:25.523820 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:25.523914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:25.523869 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:25.524037 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:25.523965 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:26.523256 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:26.523220 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:26.523721 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:26.523361 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:27.523121 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:27.523086 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:27.523337 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:27.523224 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:27.523337 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:27.523265 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:27.523874 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:27.523418 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:28.524064 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:28.524030 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:28.524462 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:28.524124 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:29.524075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:29.523884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:29.524556 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:29.523942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:29.524556 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:29.524138 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:29.524556 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:29.524204 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:29.693622 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:29.693586 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47efaae-fb25-4ea2-96af-494873864f0d" containerID="941188e0e5b69a189b290257644ae1cd47c1a68327ef89545250c754c1e7d1d5" exitCode=0 Jul 06 18:18:29.693804 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:29.693639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerDied","Data":"941188e0e5b69a189b290257644ae1cd47c1a68327ef89545250c754c1e7d1d5"} Jul 06 18:18:30.523838 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:30.523802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:30.524013 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:30.523918 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:30.698748 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:30.698713 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47efaae-fb25-4ea2-96af-494873864f0d" containerID="ceb7058603110c92facd657bb991e8c41effe3b86ded54e4b52dfd2191b20cd9" exitCode=0 Jul 06 18:18:30.699216 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:30.698760 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerDied","Data":"ceb7058603110c92facd657bb991e8c41effe3b86ded54e4b52dfd2191b20cd9"} Jul 06 18:18:31.523786 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:31.523741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:31.523786 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:31.523790 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:31.523990 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:31.523894 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:31.524026 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:31.524005 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:31.704060 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:31.704026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" event={"ID":"f47efaae-fb25-4ea2-96af-494873864f0d","Type":"ContainerStarted","Data":"bfd75788bff9cf9988cc0ded65822f3921e47fa45b60fdbd52b3b513e22b6e3b"} Jul 06 18:18:31.727461 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:31.727404 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ps2sp" podStartSLOduration=9.735301072 podStartE2EDuration="41.727388497s" totalImagesPullingTime="31.992087425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:31.727052972 +0000 UTC m=+42.873593899" watchObservedRunningTime="2026-07-06 18:18:31.727388497 +0000 UTC m=+42.873929418" Jul 06 18:18:32.523339 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:32.523293 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:32.523530 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:32.523414 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:33.523884 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:33.523850 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:33.524286 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:33.523859 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:33.524286 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:33.523957 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:33.524286 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:33.524070 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:34.523974 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:34.523935 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:34.524384 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:34.524044 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:35.523849 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:35.523817 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:35.524043 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:35.523945 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:35.524043 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:35.523991 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:35.524420 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:35.524065 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:36.523600 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:36.523556 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:36.523761 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:36.523667 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:37.523051 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:37.523013 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:37.523051 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:37.523049 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:37.523484 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:37.523146 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:37.523484 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:37.523252 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:38.523276 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:38.523241 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:38.523679 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:38.523357 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:39.524396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:39.524359 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:39.524857 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:39.524459 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:39.524857 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:39.524522 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:39.524857 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:39.524633 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:40.523448 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:40.523409 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:40.523636 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:40.523510 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:41.523786 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:41.523748 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:41.524314 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:41.523748 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:41.524314 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:41.523897 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46jsb" podUID="d5418dba-da69-4fbd-adb5-735932ed01e5" Jul 06 18:18:41.524314 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:41.523947 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s6kxp" podUID="b429876b-7071-41ab-985a-a32f0b9ea8f4" Jul 06 18:18:42.523598 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.523566 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:42.523797 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:42.523665 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4h4tn" podUID="3e58e685-1870-4dde-9709-34eb2675a948" Jul 06 18:18:42.625786 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.625758 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-250.ec2.internal" event="NodeReady" Jul 06 18:18:42.626134 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.625894 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 18:18:42.680819 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.680786 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hm4wx"] Jul 06 18:18:42.683307 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.683291 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.686268 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.686243 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b25k8\"" Jul 06 18:18:42.686956 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.686939 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 18:18:42.687708 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.687692 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 18:18:42.690993 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.690974 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rkvwj"] Jul 06 18:18:42.692871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.692856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.696507 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.696489 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 18:18:42.696609 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.696524 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 18:18:42.696609 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.696524 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 18:18:42.697030 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.697018 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mvj2\"" Jul 06 18:18:42.697072 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.697059 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 18:18:42.701176 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.701158 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hm4wx"] Jul 06 18:18:42.714356 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.714315 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rkvwj"] Jul 06 18:18:42.744013 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.743986 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47qw2\" (UniqueName: \"kubernetes.io/projected/50f70ab8-b258-480a-bc7a-eb2315a4666c-kube-api-access-47qw2\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.744183 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744025 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50f70ab8-b258-480a-bc7a-eb2315a4666c-data-volume\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.744183 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1e3dc77-a500-45c4-81d1-9c27192aeff4-metrics-tls\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.744183 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744130 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1e3dc77-a500-45c4-81d1-9c27192aeff4-config-volume\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.744390 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50f70ab8-b258-480a-bc7a-eb2315a4666c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.744390 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50f70ab8-b258-480a-bc7a-eb2315a4666c-crio-socket\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.744390 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50f70ab8-b258-480a-bc7a-eb2315a4666c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.744390 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx47c\" (UniqueName: \"kubernetes.io/projected/c1e3dc77-a500-45c4-81d1-9c27192aeff4-kube-api-access-xx47c\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.744605 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.744395 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e3dc77-a500-45c4-81d1-9c27192aeff4-tmp-dir\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.780168 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.780104 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-twqff"] Jul 06 18:18:42.781997 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.781984 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.784583 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.784557 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 18:18:42.784675 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.784606 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 18:18:42.784803 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.784789 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w2z5d\"" Jul 06 18:18:42.784886 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.784860 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 18:18:42.792237 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.792215 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-twqff"] Jul 06 18:18:42.844866 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.844830 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50f70ab8-b258-480a-bc7a-eb2315a4666c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.844874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx47c\" (UniqueName: \"kubernetes.io/projected/c1e3dc77-a500-45c4-81d1-9c27192aeff4-kube-api-access-xx47c\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.845067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.844902 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e3dc77-a500-45c4-81d1-9c27192aeff4-tmp-dir\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.845067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.844931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47qw2\" (UniqueName: \"kubernetes.io/projected/50f70ab8-b258-480a-bc7a-eb2315a4666c-kube-api-access-47qw2\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.844959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50f70ab8-b258-480a-bc7a-eb2315a4666c-data-volume\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.844983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1e3dc77-a500-45c4-81d1-9c27192aeff4-metrics-tls\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmpps\" (UniqueName: \"kubernetes.io/projected/c4854bf1-bb57-4d1f-a03d-2277f52564ff-kube-api-access-zmpps\") pod \"ingress-canary-twqff\" (UID: \"c4854bf1-bb57-4d1f-a03d-2277f52564ff\") " pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1e3dc77-a500-45c4-81d1-9c27192aeff4-config-volume\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50f70ab8-b258-480a-bc7a-eb2315a4666c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845207 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4854bf1-bb57-4d1f-a03d-2277f52564ff-cert\") pod \"ingress-canary-twqff\" (UID: \"c4854bf1-bb57-4d1f-a03d-2277f52564ff\") " pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50f70ab8-b258-480a-bc7a-eb2315a4666c-crio-socket\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845281 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50f70ab8-b258-480a-bc7a-eb2315a4666c-data-volume\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845345 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e3dc77-a500-45c4-81d1-9c27192aeff4-tmp-dir\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.845727 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845439 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50f70ab8-b258-480a-bc7a-eb2315a4666c-crio-socket\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845727 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845520 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50f70ab8-b258-480a-bc7a-eb2315a4666c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.845828 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.845735 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1e3dc77-a500-45c4-81d1-9c27192aeff4-config-volume\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.849038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.849017 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50f70ab8-b258-480a-bc7a-eb2315a4666c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.849134 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.849017 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1e3dc77-a500-45c4-81d1-9c27192aeff4-metrics-tls\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.853130 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.853105 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx47c\" (UniqueName: \"kubernetes.io/projected/c1e3dc77-a500-45c4-81d1-9c27192aeff4-kube-api-access-xx47c\") pod \"dns-default-hm4wx\" (UID: \"c1e3dc77-a500-45c4-81d1-9c27192aeff4\") " pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:42.853365 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.853345 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47qw2\" (UniqueName: \"kubernetes.io/projected/50f70ab8-b258-480a-bc7a-eb2315a4666c-kube-api-access-47qw2\") pod \"insights-runtime-extractor-rkvwj\" (UID: \"50f70ab8-b258-480a-bc7a-eb2315a4666c\") " pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:42.946136 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.946099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmpps\" (UniqueName: \"kubernetes.io/projected/c4854bf1-bb57-4d1f-a03d-2277f52564ff-kube-api-access-zmpps\") pod \"ingress-canary-twqff\" (UID: \"c4854bf1-bb57-4d1f-a03d-2277f52564ff\") " pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.946294 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.946150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4854bf1-bb57-4d1f-a03d-2277f52564ff-cert\") pod \"ingress-canary-twqff\" (UID: \"c4854bf1-bb57-4d1f-a03d-2277f52564ff\") " pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.948481 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.948461 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4854bf1-bb57-4d1f-a03d-2277f52564ff-cert\") pod \"ingress-canary-twqff\" (UID: \"c4854bf1-bb57-4d1f-a03d-2277f52564ff\") " pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.954293 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.954267 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmpps\" (UniqueName: \"kubernetes.io/projected/c4854bf1-bb57-4d1f-a03d-2277f52564ff-kube-api-access-zmpps\") pod \"ingress-canary-twqff\" (UID: \"c4854bf1-bb57-4d1f-a03d-2277f52564ff\") " pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:42.992706 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:42.992667 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:43.001111 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.001085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rkvwj" Jul 06 18:18:43.090643 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.090612 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-twqff" Jul 06 18:18:43.136979 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.136943 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rkvwj"] Jul 06 18:18:43.141565 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:43.141532 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f70ab8_b258_480a_bc7a_eb2315a4666c.slice/crio-3dd7889894e642f657be095b2c4af50667244bdd54394c6d69078e95e207476f WatchSource:0}: Error finding container 3dd7889894e642f657be095b2c4af50667244bdd54394c6d69078e95e207476f: Status 404 returned error can't find the container with id 3dd7889894e642f657be095b2c4af50667244bdd54394c6d69078e95e207476f Jul 06 18:18:43.149795 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.149772 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hm4wx"] Jul 06 18:18:43.153407 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:43.153378 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e3dc77_a500_45c4_81d1_9c27192aeff4.slice/crio-64d4358d48651ab9b56bc5f900bb648b05edcc51efce5ee01f8bc92ce4d8a180 WatchSource:0}: Error finding container 64d4358d48651ab9b56bc5f900bb648b05edcc51efce5ee01f8bc92ce4d8a180: Status 404 returned error can't find the container with id 64d4358d48651ab9b56bc5f900bb648b05edcc51efce5ee01f8bc92ce4d8a180 Jul 06 18:18:43.215833 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.215802 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-twqff"] Jul 06 18:18:43.218842 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:43.218810 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4854bf1_bb57_4d1f_a03d_2277f52564ff.slice/crio-0c16ec173e8b4f2825755b12ba45d75cfcaec236e495ed8456cfd92d11639076 WatchSource:0}: Error finding container 0c16ec173e8b4f2825755b12ba45d75cfcaec236e495ed8456cfd92d11639076: Status 404 returned error can't find the container with id 0c16ec173e8b4f2825755b12ba45d75cfcaec236e495ed8456cfd92d11639076 Jul 06 18:18:43.523578 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.523545 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:43.523578 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.523561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:43.526655 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.526628 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 18:18:43.527625 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.527532 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4lbvz\"" Jul 06 18:18:43.527625 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.527576 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 18:18:43.734421 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.734349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-twqff" event={"ID":"c4854bf1-bb57-4d1f-a03d-2277f52564ff","Type":"ContainerStarted","Data":"0c16ec173e8b4f2825755b12ba45d75cfcaec236e495ed8456cfd92d11639076"} Jul 06 18:18:43.736502 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.736413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rkvwj" event={"ID":"50f70ab8-b258-480a-bc7a-eb2315a4666c","Type":"ContainerStarted","Data":"3dd7889894e642f657be095b2c4af50667244bdd54394c6d69078e95e207476f"} Jul 06 18:18:43.738752 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:43.738681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hm4wx" event={"ID":"c1e3dc77-a500-45c4-81d1-9c27192aeff4","Type":"ContainerStarted","Data":"64d4358d48651ab9b56bc5f900bb648b05edcc51efce5ee01f8bc92ce4d8a180"} Jul 06 18:18:44.523153 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:44.523061 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:44.526037 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:44.526006 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 18:18:44.526948 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:44.526927 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m8lgm\"" Jul 06 18:18:44.527122 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:44.526933 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 18:18:45.939416 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:45.939383 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm"] Jul 06 18:18:45.950622 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:45.950584 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm"] Jul 06 18:18:45.950774 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:45.950693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:45.953190 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:45.953068 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 18:18:45.953190 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:45.953120 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-zwl8g\"" Jul 06 18:18:46.072189 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:46.072153 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c533e26-ef80-448c-ad43-0e78fae2167a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rjsgm\" (UID: \"6c533e26-ef80-448c-ad43-0e78fae2167a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:46.173083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:46.173044 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c533e26-ef80-448c-ad43-0e78fae2167a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rjsgm\" (UID: \"6c533e26-ef80-448c-ad43-0e78fae2167a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:46.173291 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:46.173223 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 18:18:46.173385 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:46.173295 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c533e26-ef80-448c-ad43-0e78fae2167a-tls-certificates podName:6c533e26-ef80-448c-ad43-0e78fae2167a nodeName:}" failed. No retries permitted until 2026-07-06 18:18:46.673276998 +0000 UTC m=+57.819817899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6c533e26-ef80-448c-ad43-0e78fae2167a-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-rjsgm" (UID: "6c533e26-ef80-448c-ad43-0e78fae2167a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 18:18:46.678216 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:46.678175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c533e26-ef80-448c-ad43-0e78fae2167a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rjsgm\" (UID: \"6c533e26-ef80-448c-ad43-0e78fae2167a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:46.680735 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:46.680704 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c533e26-ef80-448c-ad43-0e78fae2167a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rjsgm\" (UID: \"6c533e26-ef80-448c-ad43-0e78fae2167a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:46.861859 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:46.861818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:48.575278 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.575172 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm"] Jul 06 18:18:48.755048 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.755007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-twqff" event={"ID":"c4854bf1-bb57-4d1f-a03d-2277f52564ff","Type":"ContainerStarted","Data":"8013456198daa998d78d3b925c54f7b9c58d870b76ef738535eeed27de00f798"} Jul 06 18:18:48.757400 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.757370 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hm4wx" event={"ID":"c1e3dc77-a500-45c4-81d1-9c27192aeff4","Type":"ContainerStarted","Data":"c2e8f392c6570edec2288b2f0287ace6b8ddf7b538e596f9e6aea17389af2b6f"} Jul 06 18:18:48.757535 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.757422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hm4wx" event={"ID":"c1e3dc77-a500-45c4-81d1-9c27192aeff4","Type":"ContainerStarted","Data":"915c6b5e1bd5f138f8a946e9e2708c98a5092f444a9651f9e174be8fa5daa633"} Jul 06 18:18:48.757535 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.757450 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:48.758468 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.758445 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" event={"ID":"6c533e26-ef80-448c-ad43-0e78fae2167a","Type":"ContainerStarted","Data":"1c37d2bdf5e3af3deaebc391bb2867f3a73db83483c09a5f9be0485fe6025386"} Jul 06 18:18:48.771272 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.771226 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-twqff" podStartSLOduration=1.5362971970000001 podStartE2EDuration="6.771213158s" totalImagesPullingTime="5.234915961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:48.770482571 +0000 UTC m=+59.917023494" watchObservedRunningTime="2026-07-06 18:18:48.771213158 +0000 UTC m=+59.917754080" Jul 06 18:18:48.787094 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:48.787039 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hm4wx" podStartSLOduration=1.493652303 podStartE2EDuration="6.787021522s" totalImagesPullingTime="5.293369219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:48.786884142 +0000 UTC m=+59.933425060" watchObservedRunningTime="2026-07-06 18:18:48.787021522 +0000 UTC m=+59.933562423" Jul 06 18:18:49.687548 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.687520 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4ljq" Jul 06 18:18:49.764341 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.764157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" event={"ID":"6c533e26-ef80-448c-ad43-0e78fae2167a","Type":"ContainerStarted","Data":"c39d95a5c7f80096826a1a60a2926e8d431d2d60d54032c626b0de84f1bdcf52"} Jul 06 18:18:49.764513 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.764345 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:49.766541 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.766507 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rkvwj" event={"ID":"50f70ab8-b258-480a-bc7a-eb2315a4666c","Type":"ContainerStarted","Data":"7bbd289b3c36d928176d159740f1566874d1c50ca5c494e34720939f0d6bcea4"} Jul 06 18:18:49.766673 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.766573 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rkvwj" event={"ID":"50f70ab8-b258-480a-bc7a-eb2315a4666c","Type":"ContainerStarted","Data":"ac1eddb488e8ffe557405565d412e210fa1929db8139aa12fd00eabfcfa3ecad"} Jul 06 18:18:49.768993 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.768973 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" Jul 06 18:18:49.798826 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:49.798774 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rjsgm" podStartSLOduration=3.712213659 podStartE2EDuration="4.79875713s" totalImagesPullingTime="1.086543471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:49.798077692 +0000 UTC m=+60.944618614" watchObservedRunningTime="2026-07-06 18:18:49.79875713 +0000 UTC m=+60.945298051" Jul 06 18:18:50.060067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.060035 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p"] Jul 06 18:18:50.061915 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.061899 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.067216 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.067192 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 18:18:50.067216 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.067199 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 18:18:50.067437 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.067231 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 18:18:50.067437 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.067243 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 18:18:50.068213 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.068196 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s2dpv\"" Jul 06 18:18:50.068291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.068205 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 18:18:50.078478 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.078459 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p"] Jul 06 18:18:50.100918 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.100884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjrm\" (UniqueName: \"kubernetes.io/projected/a5e751c7-9dfe-422f-8d91-225bf70c29ec-kube-api-access-bpjrm\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.101034 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.100942 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.101034 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.101001 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.101140 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.101057 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e751c7-9dfe-422f-8d91-225bf70c29ec-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.205498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.205444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjrm\" (UniqueName: \"kubernetes.io/projected/a5e751c7-9dfe-422f-8d91-225bf70c29ec-kube-api-access-bpjrm\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.205498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.205499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.205741 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.205558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.205741 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:50.205675 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 18:18:50.205741 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:50.205734 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-tls podName:a5e751c7-9dfe-422f-8d91-225bf70c29ec nodeName:}" failed. No retries permitted until 2026-07-06 18:18:50.705715165 +0000 UTC m=+61.852256084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-znq4p" (UID: "a5e751c7-9dfe-422f-8d91-225bf70c29ec") : secret "prometheus-operator-tls" not found Jul 06 18:18:50.205908 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.205831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e751c7-9dfe-422f-8d91-225bf70c29ec-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.206621 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.206597 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e751c7-9dfe-422f-8d91-225bf70c29ec-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.208612 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.208593 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.221252 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.221231 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjrm\" (UniqueName: \"kubernetes.io/projected/a5e751c7-9dfe-422f-8d91-225bf70c29ec-kube-api-access-bpjrm\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.709837 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.709798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.712493 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.712465 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a5e751c7-9dfe-422f-8d91-225bf70c29ec-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-znq4p\" (UID: \"a5e751c7-9dfe-422f-8d91-225bf70c29ec\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:50.970784 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:50.970693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" Jul 06 18:18:51.108590 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:51.108558 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p"] Jul 06 18:18:52.080072 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:52.080036 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e751c7_9dfe_422f_8d91_225bf70c29ec.slice/crio-9e5c69b791783aff0d420bec74f63aaf22f5991f749e5abc094981d6f5007b46 WatchSource:0}: Error finding container 9e5c69b791783aff0d420bec74f63aaf22f5991f749e5abc094981d6f5007b46: Status 404 returned error can't find the container with id 9e5c69b791783aff0d420bec74f63aaf22f5991f749e5abc094981d6f5007b46 Jul 06 18:18:52.779807 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:52.779768 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rkvwj" event={"ID":"50f70ab8-b258-480a-bc7a-eb2315a4666c","Type":"ContainerStarted","Data":"c180f1bc79985692ec789b437bdd98c2c0d70e1865581498628c0961d8ee8ae5"} Jul 06 18:18:52.780881 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:52.780852 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" event={"ID":"a5e751c7-9dfe-422f-8d91-225bf70c29ec","Type":"ContainerStarted","Data":"9e5c69b791783aff0d420bec74f63aaf22f5991f749e5abc094981d6f5007b46"} Jul 06 18:18:53.786382 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:53.786343 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" event={"ID":"a5e751c7-9dfe-422f-8d91-225bf70c29ec","Type":"ContainerStarted","Data":"73dca77551d6d8669bbd07896958275f2cee8becfb0187e912b4db3f2fc5924c"} Jul 06 18:18:53.786817 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:53.786407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" event={"ID":"a5e751c7-9dfe-422f-8d91-225bf70c29ec","Type":"ContainerStarted","Data":"fe732252f8b9335cd46b63b1a460e825b2be237f5ab4264b229fd7e9435ec7b5"} Jul 06 18:18:53.806839 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:53.806785 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-znq4p" podStartSLOduration=2.264357598 podStartE2EDuration="3.806767282s" totalImagesPullingTime="1.542409684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:53.806531347 +0000 UTC m=+64.953072268" watchObservedRunningTime="2026-07-06 18:18:53.806767282 +0000 UTC m=+64.953308183" Jul 06 18:18:53.807695 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:53.807655 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rkvwj" podStartSLOduration=2.966628559 podStartE2EDuration="11.807640236s" totalImagesPullingTime="8.841011677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:18:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:52.801869943 +0000 UTC m=+63.948410864" watchObservedRunningTime="2026-07-06 18:18:53.807640236 +0000 UTC m=+64.954181158" Jul 06 18:18:55.148501 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.148459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:55.148926 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.148510 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:55.148926 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.148537 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:55.151058 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.151041 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 18:18:55.152034 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.152014 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 18:18:55.152034 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.152025 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 18:18:55.161560 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.161539 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5418dba-da69-4fbd-adb5-735932ed01e5-metrics-certs\") pod \"network-metrics-daemon-46jsb\" (UID: \"d5418dba-da69-4fbd-adb5-735932ed01e5\") " pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:55.161679 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.161626 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b429876b-7071-41ab-985a-a32f0b9ea8f4-original-pull-secret\") pod \"global-pull-secret-syncer-s6kxp\" (UID: \"b429876b-7071-41ab-985a-a32f0b9ea8f4\") " pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:55.161829 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.161814 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 18:18:55.172136 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.172112 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dchm4\" (UniqueName: \"kubernetes.io/projected/3e58e685-1870-4dde-9709-34eb2675a948-kube-api-access-dchm4\") pod \"network-check-target-4h4tn\" (UID: \"3e58e685-1870-4dde-9709-34eb2675a948\") " pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:55.235314 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.235275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s6kxp" Jul 06 18:18:55.244569 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.244486 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4lbvz\"" Jul 06 18:18:55.252979 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.252955 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46jsb" Jul 06 18:18:55.338080 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.338054 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m8lgm\"" Jul 06 18:18:55.346380 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.346356 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:18:55.364646 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.364616 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s6kxp"] Jul 06 18:18:55.368230 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:55.368199 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb429876b_7071_41ab_985a_a32f0b9ea8f4.slice/crio-a1232a36762fea2cd964d8ac64c398212ed31a9db4f5023bf26de2db9501bd12 WatchSource:0}: Error finding container a1232a36762fea2cd964d8ac64c398212ed31a9db4f5023bf26de2db9501bd12: Status 404 returned error can't find the container with id a1232a36762fea2cd964d8ac64c398212ed31a9db4f5023bf26de2db9501bd12 Jul 06 18:18:55.397050 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.396592 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46jsb"] Jul 06 18:18:55.401034 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:55.401007 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5418dba_da69_4fbd_adb5_735932ed01e5.slice/crio-c68c7332789cf210d5deb28f4ef9759e47f832265de2df90c730f2746395a0e6 WatchSource:0}: Error finding container c68c7332789cf210d5deb28f4ef9759e47f832265de2df90c730f2746395a0e6: Status 404 returned error can't find the container with id c68c7332789cf210d5deb28f4ef9759e47f832265de2df90c730f2746395a0e6 Jul 06 18:18:55.448181 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.448154 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk"] Jul 06 18:18:55.450873 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.450855 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.453978 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.453954 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xml2k\"" Jul 06 18:18:55.454090 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.454006 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 18:18:55.454090 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.454009 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 18:18:55.468698 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.468675 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk"] Jul 06 18:18:55.491726 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.491706 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4h4tn"] Jul 06 18:18:55.494024 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:55.494004 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e58e685_1870_4dde_9709_34eb2675a948.slice/crio-148b752ac50e90df43df770700b5e9600e3b2b84e4d5fb3fab6e7ebb25cdabb1 WatchSource:0}: Error finding container 148b752ac50e90df43df770700b5e9600e3b2b84e4d5fb3fab6e7ebb25cdabb1: Status 404 returned error can't find the container with id 148b752ac50e90df43df770700b5e9600e3b2b84e4d5fb3fab6e7ebb25cdabb1 Jul 06 18:18:55.508333 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.508294 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2"] Jul 06 18:18:55.510754 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.510734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.513672 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.513653 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 18:18:55.513672 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.513666 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 18:18:55.513922 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.513898 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 18:18:55.514012 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.513937 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p8hlf\"" Jul 06 18:18:55.535427 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.535394 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zw9rt"] Jul 06 18:18:55.537445 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.537429 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2"] Jul 06 18:18:55.537534 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.537523 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.540188 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.540168 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 18:18:55.540188 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.540185 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 18:18:55.540477 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.540463 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-445cz\"" Jul 06 18:18:55.540538 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.540524 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 18:18:55.551170 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.551147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36b5545f-2629-4e7e-9059-ec57ced9eefc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.551287 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.551187 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36b5545f-2629-4e7e-9059-ec57ced9eefc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.551287 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.551212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36b5545f-2629-4e7e-9059-ec57ced9eefc-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.551415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.551299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-479ks\" (UniqueName: \"kubernetes.io/projected/36b5545f-2629-4e7e-9059-ec57ced9eefc-kube-api-access-479ks\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.652263 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.652263 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.652263 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-229pb\" (UniqueName: \"kubernetes.io/projected/b6e11408-82ee-4e85-a707-82dda8f4e968-kube-api-access-229pb\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652263 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652248 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-accelerators-collector-config\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4644d803-d556-4c44-9834-d963e0bade34-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36b5545f-2629-4e7e-9059-ec57ced9eefc-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-tls\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-479ks\" (UniqueName: \"kubernetes.io/projected/36b5545f-2629-4e7e-9059-ec57ced9eefc-kube-api-access-479ks\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-sys\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-textfile\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-wtmp\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36b5545f-2629-4e7e-9059-ec57ced9eefc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652535 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6e11408-82ee-4e85-a707-82dda8f4e968-metrics-client-ca\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652619 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-root\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.652998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.652998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652672 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw55f\" (UniqueName: \"kubernetes.io/projected/4644d803-d556-4c44-9834-d963e0bade34-kube-api-access-hw55f\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.652998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36b5545f-2629-4e7e-9059-ec57ced9eefc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.652998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.652722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4644d803-d556-4c44-9834-d963e0bade34-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.653158 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.653006 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36b5545f-2629-4e7e-9059-ec57ced9eefc-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.654908 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.654887 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36b5545f-2629-4e7e-9059-ec57ced9eefc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.655011 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.654913 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36b5545f-2629-4e7e-9059-ec57ced9eefc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.660548 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.660527 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-479ks\" (UniqueName: \"kubernetes.io/projected/36b5545f-2629-4e7e-9059-ec57ced9eefc-kube-api-access-479ks\") pod \"openshift-state-metrics-54cf96dfcf-sxjsk\" (UID: \"36b5545f-2629-4e7e-9059-ec57ced9eefc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.753956 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.753920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.753956 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.753963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.753997 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-229pb\" (UniqueName: \"kubernetes.io/projected/b6e11408-82ee-4e85-a707-82dda8f4e968-kube-api-access-229pb\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-accelerators-collector-config\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4644d803-d556-4c44-9834-d963e0bade34-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-tls\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:55.754090 2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754151 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-sys\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:55.754173 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-tls podName:4644d803-d556-4c44-9834-d963e0bade34 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:56.254150367 +0000 UTC m=+67.400691267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-tx8z2" (UID: "4644d803-d556-4c44-9834-d963e0bade34") : secret "kube-state-metrics-tls" not found Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-sys\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754222 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:55.754199 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:55.754249 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-tls podName:b6e11408-82ee-4e85-a707-82dda8f4e968 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:56.25423285 +0000 UTC m=+67.400773749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-tls") pod "node-exporter-zw9rt" (UID: "b6e11408-82ee-4e85-a707-82dda8f4e968") : secret "node-exporter-tls" not found Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-textfile\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754294 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-wtmp\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754344 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6e11408-82ee-4e85-a707-82dda8f4e968-metrics-client-ca\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-root\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754446 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw55f\" (UniqueName: \"kubernetes.io/projected/4644d803-d556-4c44-9834-d963e0bade34-kube-api-access-hw55f\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754454 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-wtmp\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4644d803-d556-4c44-9834-d963e0bade34-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754483 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4644d803-d556-4c44-9834-d963e0bade34-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.754692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754622 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-textfile\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.755160 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754839 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-accelerators-collector-config\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.755160 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754847 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6e11408-82ee-4e85-a707-82dda8f4e968-root\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.755160 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.754839 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.755335 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.755303 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4644d803-d556-4c44-9834-d963e0bade34-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.755396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.755381 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6e11408-82ee-4e85-a707-82dda8f4e968-metrics-client-ca\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.756796 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.756774 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.756871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.756797 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.760576 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.760561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" Jul 06 18:18:55.762464 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.762431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-229pb\" (UniqueName: \"kubernetes.io/projected/b6e11408-82ee-4e85-a707-82dda8f4e968-kube-api-access-229pb\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:55.762637 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.762622 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw55f\" (UniqueName: \"kubernetes.io/projected/4644d803-d556-4c44-9834-d963e0bade34-kube-api-access-hw55f\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:55.800296 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.800248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s6kxp" event={"ID":"b429876b-7071-41ab-985a-a32f0b9ea8f4","Type":"ContainerStarted","Data":"a1232a36762fea2cd964d8ac64c398212ed31a9db4f5023bf26de2db9501bd12"} Jul 06 18:18:55.801452 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.801424 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4h4tn" event={"ID":"3e58e685-1870-4dde-9709-34eb2675a948","Type":"ContainerStarted","Data":"148b752ac50e90df43df770700b5e9600e3b2b84e4d5fb3fab6e7ebb25cdabb1"} Jul 06 18:18:55.802487 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.802466 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46jsb" event={"ID":"d5418dba-da69-4fbd-adb5-735932ed01e5","Type":"ContainerStarted","Data":"c68c7332789cf210d5deb28f4ef9759e47f832265de2df90c730f2746395a0e6"} Jul 06 18:18:55.896842 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:55.896811 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk"] Jul 06 18:18:55.899996 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:55.899968 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b5545f_2629_4e7e_9059_ec57ced9eefc.slice/crio-db182ae240e288d7e19a7a29956800263d8080a6f79fe522de00cb32ed667bee WatchSource:0}: Error finding container db182ae240e288d7e19a7a29956800263d8080a6f79fe522de00cb32ed667bee: Status 404 returned error can't find the container with id db182ae240e288d7e19a7a29956800263d8080a6f79fe522de00cb32ed667bee Jul 06 18:18:56.260163 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.260065 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-tls\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:56.261168 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.260757 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:56.265087 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.265029 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4644d803-d556-4c44-9834-d963e0bade34-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tx8z2\" (UID: \"4644d803-d556-4c44-9834-d963e0bade34\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:56.267044 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.266974 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6e11408-82ee-4e85-a707-82dda8f4e968-node-exporter-tls\") pod \"node-exporter-zw9rt\" (UID: \"b6e11408-82ee-4e85-a707-82dda8f4e968\") " pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:56.419726 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.419689 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" Jul 06 18:18:56.446426 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.446151 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zw9rt" Jul 06 18:18:56.464087 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:56.464052 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e11408_82ee_4e85_a707_82dda8f4e968.slice/crio-5815429857d72ff1f2bc12728d8c8a5f23dcbc5f03634f1650577a2a0a3ded2f WatchSource:0}: Error finding container 5815429857d72ff1f2bc12728d8c8a5f23dcbc5f03634f1650577a2a0a3ded2f: Status 404 returned error can't find the container with id 5815429857d72ff1f2bc12728d8c8a5f23dcbc5f03634f1650577a2a0a3ded2f Jul 06 18:18:56.547911 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.547877 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:18:56.551285 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.551259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.555259 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.555037 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.555816 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.556087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.556134 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.556262 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.556296 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.556432 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 18:18:56.557781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.556962 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 18:18:56.558924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.558474 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 18:18:56.558924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.558676 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6pbdt\"" Jul 06 18:18:56.571371 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.571212 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:18:56.615074 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.614982 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2"] Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.663880 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.663930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkssz\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-kube-api-access-rkssz\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.663966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.663992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-web-config\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-out\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664244 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.664473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.664278 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-volume\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.765597 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.765784 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.765784 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-web-config\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.765784 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-out\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.765784 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.765784 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766048 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.765808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-volume\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766493 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766461 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766592 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766529 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766592 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkssz\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-kube-api-access-rkssz\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766704 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766704 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766616 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766704 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766704 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.766672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.766903 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:56.766792 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 18:18:56.766903 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:56.766880 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls podName:7f10fd6e-cb90-442d-b11b-d3d03f239ed2 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:57.266831116 +0000 UTC m=+68.413372017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2") : secret "alertmanager-main-tls" not found Jul 06 18:18:56.767372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.767346 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.767801 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:18:56.767780 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle podName:7f10fd6e-cb90-442d-b11b-d3d03f239ed2 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:57.267759932 +0000 UTC m=+68.414300834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2") : configmap references non-existent config key: ca-bundle.crt Jul 06 18:18:56.768773 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.768716 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.769149 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.769123 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.769572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.769547 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-volume\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.769677 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.769602 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.770526 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.770405 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.770985 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.770961 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-web-config\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.771222 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.771171 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-out\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.771819 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.771754 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.781056 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.780984 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkssz\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-kube-api-access-rkssz\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:56.810169 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.810024 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zw9rt" event={"ID":"b6e11408-82ee-4e85-a707-82dda8f4e968","Type":"ContainerStarted","Data":"5815429857d72ff1f2bc12728d8c8a5f23dcbc5f03634f1650577a2a0a3ded2f"} Jul 06 18:18:56.812823 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.812689 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" event={"ID":"36b5545f-2629-4e7e-9059-ec57ced9eefc","Type":"ContainerStarted","Data":"48eea4825a4d4661c427e74ac86d8ed4a00d89164ac9b99b3230b4885d3f3558"} Jul 06 18:18:56.812823 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.812754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" event={"ID":"36b5545f-2629-4e7e-9059-ec57ced9eefc","Type":"ContainerStarted","Data":"70d3d4033069a6b95226b8c746cba3cbd7ab2a7ef3eb729473f39cfc0aa9ed7e"} Jul 06 18:18:56.812823 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:56.812782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" event={"ID":"36b5545f-2629-4e7e-9059-ec57ced9eefc","Type":"ContainerStarted","Data":"db182ae240e288d7e19a7a29956800263d8080a6f79fe522de00cb32ed667bee"} Jul 06 18:18:56.876690 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:56.876652 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4644d803_d556_4c44_9834_d963e0bade34.slice/crio-d121bae05bc5c429308704783d80bb0021f9b6c5a5b34ecfd861301f0471b802 WatchSource:0}: Error finding container d121bae05bc5c429308704783d80bb0021f9b6c5a5b34ecfd861301f0471b802: Status 404 returned error can't find the container with id d121bae05bc5c429308704783d80bb0021f9b6c5a5b34ecfd861301f0471b802 Jul 06 18:18:57.272931 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.272357 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:57.272931 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.272433 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:57.273887 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.273861 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:57.277283 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.277235 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:57.466128 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.465899 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:18:57.664170 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.664136 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:18:57.820701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.820614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46jsb" event={"ID":"d5418dba-da69-4fbd-adb5-735932ed01e5","Type":"ContainerStarted","Data":"5df578aedb5b44b06513066e826b1607461923ac3593327adf00c99bdf5c08dd"} Jul 06 18:18:57.820701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.820682 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46jsb" event={"ID":"d5418dba-da69-4fbd-adb5-735932ed01e5","Type":"ContainerStarted","Data":"b40ce0b08c2ab7f5cab5545b5bed3c289a4491000334d3397238e70a73fee807"} Jul 06 18:18:57.826478 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.826440 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" event={"ID":"36b5545f-2629-4e7e-9059-ec57ced9eefc","Type":"ContainerStarted","Data":"c6e993679d139a3fe3e8cf4c988d3ebddb25a20b16abf69e30d605b76434ec0f"} Jul 06 18:18:57.828231 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.828167 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" event={"ID":"4644d803-d556-4c44-9834-d963e0bade34","Type":"ContainerStarted","Data":"d121bae05bc5c429308704783d80bb0021f9b6c5a5b34ecfd861301f0471b802"} Jul 06 18:18:57.839464 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.839405 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-46jsb" podStartSLOduration=66.331586254 podStartE2EDuration="1m7.839386225s" totalImagesPullingTime="1.507799971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:57.839359645 +0000 UTC m=+68.985900568" watchObservedRunningTime="2026-07-06 18:18:57.839386225 +0000 UTC m=+68.985927147" Jul 06 18:18:57.867592 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:57.867528 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sxjsk" podStartSLOduration=1.3711624740000001 podStartE2EDuration="2.867505947s" totalImagesPullingTime="1.496343473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:57.863637748 +0000 UTC m=+69.010178673" watchObservedRunningTime="2026-07-06 18:18:57.867505947 +0000 UTC m=+69.014046870" Jul 06 18:18:57.901204 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:18:57.901166 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f10fd6e_cb90_442d_b11b_d3d03f239ed2.slice/crio-a0e17f1411291a2c8b0025ec23cba4ed26553e96de19dd505075be6e7b950a22 WatchSource:0}: Error finding container a0e17f1411291a2c8b0025ec23cba4ed26553e96de19dd505075be6e7b950a22: Status 404 returned error can't find the container with id a0e17f1411291a2c8b0025ec23cba4ed26553e96de19dd505075be6e7b950a22 Jul 06 18:18:58.553077 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.553037 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7748d77ff4-x8j2b"] Jul 06 18:18:58.556437 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.556403 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.559528 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.559503 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 18:18:58.560476 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.559825 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 18:18:58.560476 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.560009 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 18:18:58.560476 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.560014 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5bjsp\"" Jul 06 18:18:58.560476 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.560293 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3l2lorbm6c6uj\"" Jul 06 18:18:58.560476 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.560475 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 18:18:58.560765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.560480 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 18:18:58.573864 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.573837 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7748d77ff4-x8j2b"] Jul 06 18:18:58.684885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.684847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hbpr\" (UniqueName: \"kubernetes.io/projected/4510d5ec-0a37-4718-b3dc-b24cb8322e81-kube-api-access-5hbpr\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685056 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.684892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-grpc-tls\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685056 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.684959 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685056 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.685031 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.685085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.685108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-tls\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.685173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.685232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.685220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4510d5ec-0a37-4718-b3dc-b24cb8322e81-metrics-client-ca\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.769828 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.769795 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hm4wx" Jul 06 18:18:58.786371 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hbpr\" (UniqueName: \"kubernetes.io/projected/4510d5ec-0a37-4718-b3dc-b24cb8322e81-kube-api-access-5hbpr\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786371 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786377 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-grpc-tls\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786402 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786443 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-tls\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.786618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.786592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4510d5ec-0a37-4718-b3dc-b24cb8322e81-metrics-client-ca\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.787640 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.787604 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4510d5ec-0a37-4718-b3dc-b24cb8322e81-metrics-client-ca\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.789618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.789586 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.789731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.789643 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.790142 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.790122 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-grpc-tls\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.790608 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.790585 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.790889 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.790870 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-tls\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.791108 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.791086 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4510d5ec-0a37-4718-b3dc-b24cb8322e81-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.794561 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.794539 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hbpr\" (UniqueName: \"kubernetes.io/projected/4510d5ec-0a37-4718-b3dc-b24cb8322e81-kube-api-access-5hbpr\") pod \"thanos-querier-7748d77ff4-x8j2b\" (UID: \"4510d5ec-0a37-4718-b3dc-b24cb8322e81\") " pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:58.834418 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.834315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"a0e17f1411291a2c8b0025ec23cba4ed26553e96de19dd505075be6e7b950a22"} Jul 06 18:18:58.836302 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.836274 2566 generic.go:356] "Generic (PLEG): container finished" podID="b6e11408-82ee-4e85-a707-82dda8f4e968" containerID="2fec29da0faee1b5300598daaf88ae9db2bf366b2409e5a831443b4cacc127de" exitCode=0 Jul 06 18:18:58.836462 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.836313 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zw9rt" event={"ID":"b6e11408-82ee-4e85-a707-82dda8f4e968","Type":"ContainerDied","Data":"2fec29da0faee1b5300598daaf88ae9db2bf366b2409e5a831443b4cacc127de"} Jul 06 18:18:58.869528 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:58.869499 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:18:59.728550 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.728511 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-554bdb969-pb47w"] Jul 06 18:18:59.742788 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.742755 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-554bdb969-pb47w"] Jul 06 18:18:59.742943 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.742852 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.747258 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.747229 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 18:18:59.748376 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.748354 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 18:18:59.748376 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.748366 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 18:18:59.748571 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.748366 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ww6nw\"" Jul 06 18:18:59.748571 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.748393 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-85o0cdi36gi41\"" Jul 06 18:18:59.748698 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.748683 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 18:18:59.896762 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.896716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-secret-metrics-server-client-certs\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.896948 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.896786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc5pv\" (UniqueName: \"kubernetes.io/projected/ac4d2a11-08ff-4906-9ae4-70f839284f5e-kube-api-access-rc5pv\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.896948 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.896845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ac4d2a11-08ff-4906-9ae4-70f839284f5e-audit-log\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.896948 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.896932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac4d2a11-08ff-4906-9ae4-70f839284f5e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.897099 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.896957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ac4d2a11-08ff-4906-9ae4-70f839284f5e-metrics-server-audit-profiles\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.897099 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.896987 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-client-ca-bundle\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.897099 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.897075 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-secret-metrics-server-tls\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998228 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-secret-metrics-server-tls\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998228 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-secret-metrics-server-client-certs\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998531 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc5pv\" (UniqueName: \"kubernetes.io/projected/ac4d2a11-08ff-4906-9ae4-70f839284f5e-kube-api-access-rc5pv\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998531 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998303 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ac4d2a11-08ff-4906-9ae4-70f839284f5e-audit-log\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac4d2a11-08ff-4906-9ae4-70f839284f5e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ac4d2a11-08ff-4906-9ae4-70f839284f5e-metrics-server-audit-profiles\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-client-ca-bundle\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.998885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.998786 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ac4d2a11-08ff-4906-9ae4-70f839284f5e-audit-log\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.999494 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.999470 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac4d2a11-08ff-4906-9ae4-70f839284f5e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:18:59.999814 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:18:59.999788 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ac4d2a11-08ff-4906-9ae4-70f839284f5e-metrics-server-audit-profiles\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:00.001718 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.001410 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-secret-metrics-server-client-certs\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:00.001718 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.001636 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-client-ca-bundle\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:00.002727 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.002692 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ac4d2a11-08ff-4906-9ae4-70f839284f5e-secret-metrics-server-tls\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:00.008206 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.008163 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc5pv\" (UniqueName: \"kubernetes.io/projected/ac4d2a11-08ff-4906-9ae4-70f839284f5e-kube-api-access-rc5pv\") pod \"metrics-server-554bdb969-pb47w\" (UID: \"ac4d2a11-08ff-4906-9ae4-70f839284f5e\") " pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:00.053104 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.053058 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:00.240077 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.240043 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc"] Jul 06 18:19:00.255051 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.254961 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc"] Jul 06 18:19:00.255207 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.255085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.259258 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.259105 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 18:19:00.259258 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.259117 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-ks8d5\"" Jul 06 18:19:00.401879 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.401838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4gc4\" (UniqueName: \"kubernetes.io/projected/41455cdd-d221-4a01-91ff-3d5e428d17ad-kube-api-access-z4gc4\") pod \"monitoring-plugin-d5dcff7cf-p8rkc\" (UID: \"41455cdd-d221-4a01-91ff-3d5e428d17ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.402061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.401922 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41455cdd-d221-4a01-91ff-3d5e428d17ad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-p8rkc\" (UID: \"41455cdd-d221-4a01-91ff-3d5e428d17ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.503504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.503437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41455cdd-d221-4a01-91ff-3d5e428d17ad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-p8rkc\" (UID: \"41455cdd-d221-4a01-91ff-3d5e428d17ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.503900 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.503836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4gc4\" (UniqueName: \"kubernetes.io/projected/41455cdd-d221-4a01-91ff-3d5e428d17ad-kube-api-access-z4gc4\") pod \"monitoring-plugin-d5dcff7cf-p8rkc\" (UID: \"41455cdd-d221-4a01-91ff-3d5e428d17ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.506906 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.506848 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41455cdd-d221-4a01-91ff-3d5e428d17ad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-p8rkc\" (UID: \"41455cdd-d221-4a01-91ff-3d5e428d17ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.524816 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.524783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4gc4\" (UniqueName: \"kubernetes.io/projected/41455cdd-d221-4a01-91ff-3d5e428d17ad-kube-api-access-z4gc4\") pod \"monitoring-plugin-d5dcff7cf-p8rkc\" (UID: \"41455cdd-d221-4a01-91ff-3d5e428d17ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:00.569192 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:00.569156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:01.724466 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.724397 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7748d77ff4-x8j2b"] Jul 06 18:19:01.756826 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.756806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc"] Jul 06 18:19:01.763897 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:19:01.763135 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41455cdd_d221_4a01_91ff_3d5e428d17ad.slice/crio-5b0a45ab9a0d7c885b3f1d88aac4028972dd175bf1db23bc48f553e94b849afe WatchSource:0}: Error finding container 5b0a45ab9a0d7c885b3f1d88aac4028972dd175bf1db23bc48f553e94b849afe: Status 404 returned error can't find the container with id 5b0a45ab9a0d7c885b3f1d88aac4028972dd175bf1db23bc48f553e94b849afe Jul 06 18:19:01.789094 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.789015 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-554bdb969-pb47w"] Jul 06 18:19:01.857347 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.857170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zw9rt" event={"ID":"b6e11408-82ee-4e85-a707-82dda8f4e968","Type":"ContainerStarted","Data":"881b66c583e5d931a24e4d8e3b67099a9d4b93a2a7d2bdd13b025dc91c699031"} Jul 06 18:19:01.857347 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.857270 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zw9rt" event={"ID":"b6e11408-82ee-4e85-a707-82dda8f4e968","Type":"ContainerStarted","Data":"70d8f79ef983d59d65258fdf04ab3fa4002861b32ba15b0216913192fb0d5d24"} Jul 06 18:19:01.859228 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.859185 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"bde88716664bb4fb98f35acba2e64c5855e0d2efa3cd10f7af871a05285930ba"} Jul 06 18:19:01.863470 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.863437 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" event={"ID":"4644d803-d556-4c44-9834-d963e0bade34","Type":"ContainerStarted","Data":"6ceed6c8f32d7e2fd1a6aae87f469caa66a808ecea6852a329e47485262a6110"} Jul 06 18:19:01.863624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.863501 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" event={"ID":"4644d803-d556-4c44-9834-d963e0bade34","Type":"ContainerStarted","Data":"a2881c3e657b9de3347ecb3a22487da474f249df9fe5d2870e5a0e1adeb47dee"} Jul 06 18:19:01.874373 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.874266 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" event={"ID":"41455cdd-d221-4a01-91ff-3d5e428d17ad","Type":"ContainerStarted","Data":"5b0a45ab9a0d7c885b3f1d88aac4028972dd175bf1db23bc48f553e94b849afe"} Jul 06 18:19:01.877163 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.877124 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" event={"ID":"ac4d2a11-08ff-4906-9ae4-70f839284f5e","Type":"ContainerStarted","Data":"206739acd5a64d69fb02c71aabe6de2516f429b08ffcc50f084b9d58ab115400"} Jul 06 18:19:01.879179 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.879099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s6kxp" event={"ID":"b429876b-7071-41ab-985a-a32f0b9ea8f4","Type":"ContainerStarted","Data":"2f353cc677168c81b9c5645e4102f7a66423285a4de8660cca9dd0f4d0b3fe2b"} Jul 06 18:19:01.883089 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.882203 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc" exitCode=0 Jul 06 18:19:01.883089 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.882302 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc"} Jul 06 18:19:01.898515 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.897495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4h4tn" event={"ID":"3e58e685-1870-4dde-9709-34eb2675a948","Type":"ContainerStarted","Data":"015fcc242e61e2b6aed0417610c5d3526ece49d0d049475943cb5a580455452d"} Jul 06 18:19:01.898515 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.898090 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:19:01.898515 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.898363 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:19:01.905557 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.905530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:01.915720 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.915404 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 18:19:01.915720 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.915452 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 18:19:01.915720 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.915405 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 18:19:01.915720 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.915655 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 18:19:01.916085 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.915878 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 18:19:01.916568 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.916409 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 18:19:01.917268 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.917112 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 18:19:01.917489 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.917476 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 18:19:01.918778 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.918760 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 18:19:01.924035 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.919200 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4teih7ntrasis\"" Jul 06 18:19:01.924269 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.919261 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 18:19:01.924422 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.919301 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 18:19:01.924922 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.919746 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 18:19:01.924922 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.919796 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pbmn4\"" Jul 06 18:19:01.924922 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.920706 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 18:19:01.932492 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.932464 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 18:19:01.957930 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.957858 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zw9rt" podStartSLOduration=5.403481774 podStartE2EDuration="6.957837435s" totalImagesPullingTime="1.467002055s" totalInitContainerRuntime="87.353606ms" isStatefulPod=true podCreationTimestamp="2026-07-06 18:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:01.916894664 +0000 UTC m=+73.063435588" watchObservedRunningTime="2026-07-06 18:19:01.957837435 +0000 UTC m=+73.104378355" Jul 06 18:19:01.958503 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.958479 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:19:01.998186 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:01.998067 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4h4tn" podStartSLOduration=66.011264498 podStartE2EDuration="1m11.998036403s" totalImagesPullingTime="5.986771905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:01.997597984 +0000 UTC m=+73.144138904" watchObservedRunningTime="2026-07-06 18:19:01.998036403 +0000 UTC m=+73.144577325" Jul 06 18:19:02.000259 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.000187 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s6kxp" podStartSLOduration=65.88720845 podStartE2EDuration="1m12.000166451s" totalImagesPullingTime="6.112958001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:01.961893088 +0000 UTC m=+73.108434010" watchObservedRunningTime="2026-07-06 18:19:02.000166451 +0000 UTC m=+73.146707400" Jul 06 18:19:02.022718 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.022681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnqxb\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-kube-api-access-pnqxb\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.022863 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.022738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.022863 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.022812 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.022927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023307 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023461 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023611 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023640 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023739 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.024717 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.023805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.124996 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.124948 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.125194 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.125014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.125194 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.125053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.125194 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.125087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.125194 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.125123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.125194 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.125150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.125846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.125819 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.126143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.126120 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.126736 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.126717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.126903 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.126887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127483 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127572 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127561 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127662 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127747 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127838 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.127935 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127924 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.128018 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.128006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.128123 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.128112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.128216 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.128193 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.128277 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.128210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnqxb\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-kube-api-access-pnqxb\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.128539 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.128513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.128758 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.128650 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.129185 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.129162 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.129289 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.127948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.129382 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.129295 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.130270 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.130101 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.130270 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.130138 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.130441 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.130280 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.130441 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.130301 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.130850 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.130806 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.131104 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.131082 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.131405 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.131373 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.131813 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.131790 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.132164 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.132125 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.132424 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.132402 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.132508 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.132472 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.133266 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.133240 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.133649 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.133632 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.139579 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.139553 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnqxb\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-kube-api-access-pnqxb\") pod \"prometheus-k8s-0\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.238337 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.238269 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:02.388599 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.388564 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:19:02.392254 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:19:02.392224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2f5426_636c_4e9c_a2d2_2a362a427f85.slice/crio-b9acac467525301bc7fb5f5cdbd6466be217c4b0016c2455caa85ecc3f884b2e WatchSource:0}: Error finding container b9acac467525301bc7fb5f5cdbd6466be217c4b0016c2455caa85ecc3f884b2e: Status 404 returned error can't find the container with id b9acac467525301bc7fb5f5cdbd6466be217c4b0016c2455caa85ecc3f884b2e Jul 06 18:19:02.907018 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.906837 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="1f6e7a212c447a52aca00b1b643f78b1dfdb5ef28623ebc489d0526ff5502e94" exitCode=0 Jul 06 18:19:02.907018 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.906950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"1f6e7a212c447a52aca00b1b643f78b1dfdb5ef28623ebc489d0526ff5502e94"} Jul 06 18:19:02.910253 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.907088 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"b9acac467525301bc7fb5f5cdbd6466be217c4b0016c2455caa85ecc3f884b2e"} Jul 06 18:19:02.918047 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.917926 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" event={"ID":"4644d803-d556-4c44-9834-d963e0bade34","Type":"ContainerStarted","Data":"7611c12b2702a2c2e17e8ad0ffbf5be6f42d41bc09afe38c4f9f57431e9ee882"} Jul 06 18:19:02.978486 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:02.978426 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tx8z2" podStartSLOduration=3.376121104 podStartE2EDuration="7.978402005s" totalImagesPullingTime="4.602280901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:02.977507019 +0000 UTC m=+74.124048037" watchObservedRunningTime="2026-07-06 18:19:02.978402005 +0000 UTC m=+74.124942961" Jul 06 18:19:05.936064 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.935971 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"b5e82bc255b76cb854967bc73177d92d97207644729ab0775b5a073fb5f31a68"} Jul 06 18:19:05.936064 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.936051 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"2f461a8190e08a6564fa9a5c88c0155d43f4e9dd2562b356e5f666e49392b7f8"} Jul 06 18:19:05.936564 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.936094 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"be3bc020a63173fcc01501ff0c5d14e110fda35167e9d3e23759740de246eae8"} Jul 06 18:19:05.937692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.937659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" event={"ID":"41455cdd-d221-4a01-91ff-3d5e428d17ad","Type":"ContainerStarted","Data":"aa0a4bb1274a41c23b8db696184ac870f34962dbdecdcdc43cd25c528ce065ad"} Jul 06 18:19:05.937885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.937865 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:05.939488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.939462 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" event={"ID":"ac4d2a11-08ff-4906-9ae4-70f839284f5e","Type":"ContainerStarted","Data":"2be76deadba757cd631fb7a5953d6b4c68e724e123f8e9a61b3733b9aa60b581"} Jul 06 18:19:05.943456 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.943432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e"} Jul 06 18:19:05.943561 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.943500 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7"} Jul 06 18:19:05.943561 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.943546 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6"} Jul 06 18:19:05.943686 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.943587 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565"} Jul 06 18:19:05.943686 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.943627 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc"} Jul 06 18:19:05.944309 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.944289 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" Jul 06 18:19:05.963970 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.963925 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p8rkc" podStartSLOduration=2.431643224 podStartE2EDuration="5.963909319s" totalImagesPullingTime="3.532266095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:05.962539539 +0000 UTC m=+77.109080463" watchObservedRunningTime="2026-07-06 18:19:05.963909319 +0000 UTC m=+77.110450218" Jul 06 18:19:05.985573 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:05.985521 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" podStartSLOduration=3.495687463 podStartE2EDuration="6.985501518s" totalImagesPullingTime="3.489814055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:05.985169699 +0000 UTC m=+77.131710625" watchObservedRunningTime="2026-07-06 18:19:05.985501518 +0000 UTC m=+77.132042442" Jul 06 18:19:07.955988 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.955957 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerStarted","Data":"54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a"} Jul 06 18:19:07.959255 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.959229 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"b8b65c60e0ba7decdab36411b7d5367224690e73c3ee466a1c0893118622ea67"} Jul 06 18:19:07.959415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.959296 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"2e4da991916f01e7766d9a25a16a6ef5240de8fc3e63d8e051215898784bf9d1"} Jul 06 18:19:07.959415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.959352 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"82ba32968c6cf09a32bbb472e1dff6a39cf0df40dc8127a936e460f0bd783f45"} Jul 06 18:19:07.959415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.959388 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"93212c8ddb7a1f888f1325f725a208f0476bab64a931bfcbd11455a429c3ec93"} Jul 06 18:19:07.959530 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.959422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"b5957533836b4c2b2cf216ed566543c784b3f6c93eb58f6b062f23a4ea03964e"} Jul 06 18:19:07.959530 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.959450 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerStarted","Data":"9edd2250e64de441f244f15fb10aae321a2f18e0a348ccc55111510715a12f62"} Jul 06 18:19:07.962418 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.962395 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"474a37863e161fefb11b17e737041d79ffb631d21471dd210a34f6671d5c3fd3"} Jul 06 18:19:07.962507 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.962449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"95db23155fe088a8932c069badbe248cbed2ddd92a6ead79564e8c15699916ea"} Jul 06 18:19:07.962507 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.962480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" event={"ID":"4510d5ec-0a37-4718-b3dc-b24cb8322e81","Type":"ContainerStarted","Data":"62d6d76b70ec15db94349caab0e943ff8623812ee41e6b6e934622744ef6d4d7"} Jul 06 18:19:07.983781 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:07.983726 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.027411893 podStartE2EDuration="11.983709759s" totalImagesPullingTime="8.930366787s" totalInitContainerRuntime="25.931079ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:07.982920783 +0000 UTC m=+79.129461748" watchObservedRunningTime="2026-07-06 18:19:07.983709759 +0000 UTC m=+79.130250692" Jul 06 18:19:08.008372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:08.008290 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" podStartSLOduration=4.464205746 podStartE2EDuration="10.008267479s" totalImagesPullingTime="5.544061733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:08.00641139 +0000 UTC m=+79.152952313" watchObservedRunningTime="2026-07-06 18:19:08.008267479 +0000 UTC m=+79.154808402" Jul 06 18:19:08.033584 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:08.033364 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.927353167 podStartE2EDuration="7.033341799s" totalImagesPullingTime="4.081477338s" totalInitContainerRuntime="24.511294ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:08.032349811 +0000 UTC m=+79.178890735" watchObservedRunningTime="2026-07-06 18:19:08.033341799 +0000 UTC m=+79.179882715" Jul 06 18:19:08.966918 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:08.966887 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:19:12.238915 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:12.238790 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:19:14.978679 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:14.978650 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7748d77ff4-x8j2b" Jul 06 18:19:20.053943 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:20.053903 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:20.054407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:20.053970 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:33.926216 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:33.926175 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4h4tn" Jul 06 18:19:35.283692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:35.283663 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/init-config-reloader/0.log" Jul 06 18:19:35.289762 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:35.289741 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/alertmanager/0.log" Jul 06 18:19:35.385654 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:35.385628 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/config-reloader/0.log" Jul 06 18:19:35.585706 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:35.585627 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/kube-rbac-proxy-web/0.log" Jul 06 18:19:35.786150 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:35.786118 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/kube-rbac-proxy/0.log" Jul 06 18:19:35.985875 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:35.985851 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/kube-rbac-proxy-metric/0.log" Jul 06 18:19:36.184800 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:36.184769 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7f10fd6e-cb90-442d-b11b-d3d03f239ed2/prom-label-proxy/0.log" Jul 06 18:19:36.585019 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:36.584985 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tx8z2_4644d803-d556-4c44-9834-d963e0bade34/kube-state-metrics/0.log" Jul 06 18:19:36.785038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:36.785007 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tx8z2_4644d803-d556-4c44-9834-d963e0bade34/kube-rbac-proxy-main/0.log" Jul 06 18:19:36.985668 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:36.985633 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tx8z2_4644d803-d556-4c44-9834-d963e0bade34/kube-rbac-proxy-self/0.log" Jul 06 18:19:37.184960 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:37.184930 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-554bdb969-pb47w_ac4d2a11-08ff-4906-9ae4-70f839284f5e/metrics-server/0.log" Jul 06 18:19:37.385595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:37.385514 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-p8rkc_41455cdd-d221-4a01-91ff-3d5e428d17ad/monitoring-plugin/0.log" Jul 06 18:19:38.784902 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:38.784874 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zw9rt_b6e11408-82ee-4e85-a707-82dda8f4e968/init-textfile/0.log" Jul 06 18:19:38.986497 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:38.986462 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zw9rt_b6e11408-82ee-4e85-a707-82dda8f4e968/node-exporter/0.log" Jul 06 18:19:39.185771 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:39.185692 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zw9rt_b6e11408-82ee-4e85-a707-82dda8f4e968/kube-rbac-proxy/0.log" Jul 06 18:19:39.385801 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:39.385771 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sxjsk_36b5545f-2629-4e7e-9059-ec57ced9eefc/kube-rbac-proxy-main/0.log" Jul 06 18:19:39.585473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:39.585444 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sxjsk_36b5545f-2629-4e7e-9059-ec57ced9eefc/kube-rbac-proxy-self/0.log" Jul 06 18:19:39.784853 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:39.784821 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sxjsk_36b5545f-2629-4e7e-9059-ec57ced9eefc/openshift-state-metrics/0.log" Jul 06 18:19:39.985105 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:39.985073 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/init-config-reloader/0.log" Jul 06 18:19:40.058765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.058736 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:40.062564 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.062544 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-554bdb969-pb47w" Jul 06 18:19:40.185770 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.185743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/prometheus/0.log" Jul 06 18:19:40.397448 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.397368 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/config-reloader/0.log" Jul 06 18:19:40.585728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.585703 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/thanos-sidecar/0.log" Jul 06 18:19:40.787332 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.787292 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/kube-rbac-proxy-web/0.log" Jul 06 18:19:40.985798 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:40.985760 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/kube-rbac-proxy/0.log" Jul 06 18:19:41.185518 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:41.185444 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a2f5426-636c-4e9c-a2d2-2a362a427f85/kube-rbac-proxy-thanos/0.log" Jul 06 18:19:41.386336 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:41.386302 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-znq4p_a5e751c7-9dfe-422f-8d91-225bf70c29ec/prometheus-operator/0.log" Jul 06 18:19:41.584778 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:41.584749 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-znq4p_a5e751c7-9dfe-422f-8d91-225bf70c29ec/kube-rbac-proxy/0.log" Jul 06 18:19:41.784838 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:41.784808 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-rjsgm_6c533e26-ef80-448c-ad43-0e78fae2167a/prometheus-operator-admission-webhook/0.log" Jul 06 18:19:41.986830 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:41.986797 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/thanos-query/0.log" Jul 06 18:19:42.185096 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:42.185064 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy-web/0.log" Jul 06 18:19:42.385014 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:42.384935 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy/0.log" Jul 06 18:19:42.584940 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:42.584908 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/prom-label-proxy/0.log" Jul 06 18:19:42.785373 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:42.785346 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy-rules/0.log" Jul 06 18:19:42.984702 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:19:42.984668 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy-metrics/0.log" Jul 06 18:20:02.239543 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:02.239504 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:02.259604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:02.259575 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:03.209862 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:03.209827 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:15.714373 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714335 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:15.714859 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714622 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="alertmanager" containerID="cri-o://48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc" gracePeriod=120 Jul 06 18:20:15.714859 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714719 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy" containerID="cri-o://816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7" gracePeriod=120 Jul 06 18:20:15.714859 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714711 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-metric" containerID="cri-o://994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e" gracePeriod=120 Jul 06 18:20:15.714859 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714743 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="config-reloader" containerID="cri-o://60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565" gracePeriod=120 Jul 06 18:20:15.714859 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714784 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="prom-label-proxy" containerID="cri-o://54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a" gracePeriod=120 Jul 06 18:20:15.715103 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:15.714731 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-web" containerID="cri-o://80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6" gracePeriod=120 Jul 06 18:20:16.249788 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249753 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a" exitCode=0 Jul 06 18:20:16.249788 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249778 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e" exitCode=0 Jul 06 18:20:16.249788 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249785 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7" exitCode=0 Jul 06 18:20:16.249788 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249790 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565" exitCode=0 Jul 06 18:20:16.249788 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249795 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc" exitCode=0 Jul 06 18:20:16.250086 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249828 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a"} Jul 06 18:20:16.250086 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249925 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e"} Jul 06 18:20:16.250086 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249953 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7"} Jul 06 18:20:16.250086 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.249977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565"} Jul 06 18:20:16.250086 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.250000 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc"} Jul 06 18:20:16.948812 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.948791 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:16.983376 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983346 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-volume\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.983376 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983384 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.983624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983416 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-main-db\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.983725 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983695 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983802 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983867 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-out\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983899 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983939 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-tls-assets\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.983984 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-web-config\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984022 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984059 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984090 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkssz\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-kube-api-access-rkssz\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984117 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-cluster-tls-config\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984139 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-metrics-client-ca\") pod \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\" (UID: \"7f10fd6e-cb90-442d-b11b-d3d03f239ed2\") " Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984444 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-main-db\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:16.984981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.984791 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:16.986551 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.986509 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:16.987177 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.987149 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:16.987524 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.987497 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:16.988871 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.988832 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-volume" (OuterVolumeSpecName: "config-volume") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:16.989335 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.989282 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:16.990513 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.990487 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:20:16.990673 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.990647 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-kube-api-access-rkssz" (OuterVolumeSpecName: "kube-api-access-rkssz") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "kube-api-access-rkssz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:20:16.990793 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.990717 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:16.991270 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.991214 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-out" (OuterVolumeSpecName: "config-out") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:20:16.996676 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:16.996649 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:17.003061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.003035 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-web-config" (OuterVolumeSpecName: "web-config") pod "7f10fd6e-cb90-442d-b11b-d3d03f239ed2" (UID: "7f10fd6e-cb90-442d-b11b-d3d03f239ed2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085246 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-web-config\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085276 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085288 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085298 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkssz\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-kube-api-access-rkssz\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085307 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-cluster-tls-config\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085339 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-metrics-client-ca\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085348 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-volume\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085352 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085356 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-main-tls\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085364 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085374 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-config-out\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085382 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.085701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.085390 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f10fd6e-cb90-442d-b11b-d3d03f239ed2-tls-assets\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:17.257798 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.257762 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerID="80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6" exitCode=0 Jul 06 18:20:17.257981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.257848 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6"} Jul 06 18:20:17.257981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.257924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f10fd6e-cb90-442d-b11b-d3d03f239ed2","Type":"ContainerDied","Data":"a0e17f1411291a2c8b0025ec23cba4ed26553e96de19dd505075be6e7b950a22"} Jul 06 18:20:17.257981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.257856 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.257981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.257957 2566 scope.go:117] "RemoveContainer" containerID="54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a" Jul 06 18:20:17.266199 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.266180 2566 scope.go:117] "RemoveContainer" containerID="994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e" Jul 06 18:20:17.272942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.272919 2566 scope.go:117] "RemoveContainer" containerID="816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7" Jul 06 18:20:17.281411 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.281285 2566 scope.go:117] "RemoveContainer" containerID="80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6" Jul 06 18:20:17.282247 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.282227 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:17.288465 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.288443 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:17.288919 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.288905 2566 scope.go:117] "RemoveContainer" containerID="60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565" Jul 06 18:20:17.295629 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.295614 2566 scope.go:117] "RemoveContainer" containerID="48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc" Jul 06 18:20:17.302827 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.302811 2566 scope.go:117] "RemoveContainer" containerID="446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc" Jul 06 18:20:17.310360 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.310336 2566 scope.go:117] "RemoveContainer" containerID="54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a" Jul 06 18:20:17.310912 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.310758 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a\": container with ID starting with 54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a not found: ID does not exist" containerID="54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a" Jul 06 18:20:17.310912 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.310795 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a"} err="failed to get container status \"54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a\": rpc error: code = NotFound desc = could not find container \"54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a\": container with ID starting with 54ee1c1947f92f365f4e1c20c591a57b25d1fb362f84d6b1b2cf6137ffdde82a not found: ID does not exist" Jul 06 18:20:17.310912 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.310836 2566 scope.go:117] "RemoveContainer" containerID="994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e" Jul 06 18:20:17.311246 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.311114 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e\": container with ID starting with 994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e not found: ID does not exist" containerID="994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e" Jul 06 18:20:17.311246 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.311146 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e"} err="failed to get container status \"994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e\": rpc error: code = NotFound desc = could not find container \"994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e\": container with ID starting with 994119d40342e9c7389886551fae81d8ed465c6fc01ae122747d76bc7571f00e not found: ID does not exist" Jul 06 18:20:17.311246 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.311189 2566 scope.go:117] "RemoveContainer" containerID="816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7" Jul 06 18:20:17.311570 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.311502 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7\": container with ID starting with 816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7 not found: ID does not exist" containerID="816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7" Jul 06 18:20:17.311570 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.311530 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7"} err="failed to get container status \"816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7\": rpc error: code = NotFound desc = could not find container \"816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7\": container with ID starting with 816d577ab509954610668e018c45ad92f0d19a6cb7b77d73035dd144dbf313c7 not found: ID does not exist" Jul 06 18:20:17.311570 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.311552 2566 scope.go:117] "RemoveContainer" containerID="80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6" Jul 06 18:20:17.311833 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.311816 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6\": container with ID starting with 80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6 not found: ID does not exist" containerID="80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6" Jul 06 18:20:17.311938 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.311839 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6"} err="failed to get container status \"80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6\": rpc error: code = NotFound desc = could not find container \"80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6\": container with ID starting with 80abb7ae2f7341b7667668b68094a2f88149d8312ce35fce246fd1d30e9871c6 not found: ID does not exist" Jul 06 18:20:17.311938 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.311853 2566 scope.go:117] "RemoveContainer" containerID="60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565" Jul 06 18:20:17.312137 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.312113 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565\": container with ID starting with 60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565 not found: ID does not exist" containerID="60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565" Jul 06 18:20:17.312179 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.312147 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565"} err="failed to get container status \"60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565\": rpc error: code = NotFound desc = could not find container \"60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565\": container with ID starting with 60f1b3acde422203bbef61b3d63ed16acf7b696d797dd869223bf8d6af05f565 not found: ID does not exist" Jul 06 18:20:17.312179 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.312168 2566 scope.go:117] "RemoveContainer" containerID="48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc" Jul 06 18:20:17.312491 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.312442 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc\": container with ID starting with 48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc not found: ID does not exist" containerID="48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc" Jul 06 18:20:17.312491 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.312470 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc"} err="failed to get container status \"48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc\": rpc error: code = NotFound desc = could not find container \"48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc\": container with ID starting with 48e125104b519de639beba2fe16d692d8096cb45ffa59025a2b46c7da80e67dc not found: ID does not exist" Jul 06 18:20:17.312491 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.312492 2566 scope.go:117] "RemoveContainer" containerID="446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc" Jul 06 18:20:17.312764 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.312747 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:17.312847 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:20:17.312745 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc\": container with ID starting with 446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc not found: ID does not exist" containerID="446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc" Jul 06 18:20:17.312912 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.312852 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc"} err="failed to get container status \"446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc\": rpc error: code = NotFound desc = could not find container \"446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc\": container with ID starting with 446e2acc59203ac4edb870093c34cb3d3d432844c0b21be87a27bb1abb8df0cc not found: ID does not exist" Jul 06 18:20:17.313058 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313047 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-web" Jul 06 18:20:17.313092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313061 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-web" Jul 06 18:20:17.313092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313075 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="alertmanager" Jul 06 18:20:17.313092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313081 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="alertmanager" Jul 06 18:20:17.313092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313092 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy" Jul 06 18:20:17.313215 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313171 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy" Jul 06 18:20:17.313248 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313216 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-metric" Jul 06 18:20:17.313291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313253 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-metric" Jul 06 18:20:17.313291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313274 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="config-reloader" Jul 06 18:20:17.313291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313280 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="config-reloader" Jul 06 18:20:17.313291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313288 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="init-config-reloader" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313294 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="init-config-reloader" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313304 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="prom-label-proxy" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313310 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="prom-label-proxy" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313398 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-web" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313406 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="prom-label-proxy" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313415 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="config-reloader" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313424 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="alertmanager" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313431 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy" Jul 06 18:20:17.313488 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.313437 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" containerName="kube-rbac-proxy-metric" Jul 06 18:20:17.318846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.318830 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.321184 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321151 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 18:20:17.321184 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321179 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 18:20:17.321381 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321174 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 18:20:17.321381 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321197 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 18:20:17.321381 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321160 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 18:20:17.321544 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321412 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 18:20:17.321544 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321445 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 18:20:17.321891 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.321877 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6pbdt\"" Jul 06 18:20:17.322036 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.322021 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 18:20:17.326207 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.326188 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 18:20:17.330461 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.330440 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:17.387547 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9b71495-bba2-4605-a416-1f886b718fc6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387547 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387499 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9b71495-bba2-4605-a416-1f886b718fc6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387547 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387524 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9b71495-bba2-4605-a416-1f886b718fc6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387576 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387597 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c9b71495-bba2-4605-a416-1f886b718fc6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-config-volume\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9b71495-bba2-4605-a416-1f886b718fc6-config-out\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-web-config\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.387765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmscz\" (UniqueName: \"kubernetes.io/projected/c9b71495-bba2-4605-a416-1f886b718fc6-kube-api-access-xmscz\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.388060 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.388060 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.387858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488238 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488202 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmscz\" (UniqueName: \"kubernetes.io/projected/c9b71495-bba2-4605-a416-1f886b718fc6-kube-api-access-xmscz\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488238 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9b71495-bba2-4605-a416-1f886b718fc6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9b71495-bba2-4605-a416-1f886b718fc6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9b71495-bba2-4605-a416-1f886b718fc6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c9b71495-bba2-4605-a416-1f886b718fc6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-config-volume\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488917 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488522 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9b71495-bba2-4605-a416-1f886b718fc6-config-out\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.488917 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488543 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-web-config\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.489014 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.488959 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c9b71495-bba2-4605-a416-1f886b718fc6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.489516 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.489180 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9b71495-bba2-4605-a416-1f886b718fc6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.489516 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.489312 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9b71495-bba2-4605-a416-1f886b718fc6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.491560 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.491469 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.491560 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.491476 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9b71495-bba2-4605-a416-1f886b718fc6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.491560 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.491528 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.491801 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.491570 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.491801 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.491671 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-web-config\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.491899 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.491815 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.492092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.492074 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.492161 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.492106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c9b71495-bba2-4605-a416-1f886b718fc6-config-volume\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.493426 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.493406 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9b71495-bba2-4605-a416-1f886b718fc6-config-out\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.496578 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.496561 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmscz\" (UniqueName: \"kubernetes.io/projected/c9b71495-bba2-4605-a416-1f886b718fc6-kube-api-access-xmscz\") pod \"alertmanager-main-0\" (UID: \"c9b71495-bba2-4605-a416-1f886b718fc6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.528279 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.528249 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f10fd6e-cb90-442d-b11b-d3d03f239ed2" path="/var/lib/kubelet/pods/7f10fd6e-cb90-442d-b11b-d3d03f239ed2/volumes" Jul 06 18:20:17.630789 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.630764 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:17.766498 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:17.766468 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:17.769557 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:20:17.769532 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b71495_bba2_4605_a416_1f886b718fc6.slice/crio-ebbb366ff7dcafe389ee30cfd76de29a7f918b75fa361a7f1dcafbf96053476d WatchSource:0}: Error finding container ebbb366ff7dcafe389ee30cfd76de29a7f918b75fa361a7f1dcafbf96053476d: Status 404 returned error can't find the container with id ebbb366ff7dcafe389ee30cfd76de29a7f918b75fa361a7f1dcafbf96053476d Jul 06 18:20:18.263471 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:18.263437 2566 generic.go:356] "Generic (PLEG): container finished" podID="c9b71495-bba2-4605-a416-1f886b718fc6" containerID="17c4532009682ad60638d3f0fb07643f50a2de54dc639eb388840f392e527dbb" exitCode=0 Jul 06 18:20:18.263933 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:18.263522 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerDied","Data":"17c4532009682ad60638d3f0fb07643f50a2de54dc639eb388840f392e527dbb"} Jul 06 18:20:18.263933 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:18.263592 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"ebbb366ff7dcafe389ee30cfd76de29a7f918b75fa361a7f1dcafbf96053476d"} Jul 06 18:20:19.271459 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.271423 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"22f73235de7223279c728cd5bcef6ca423241a5481318b06a442f2d55f508c41"} Jul 06 18:20:19.271830 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.271516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"b79f62707278a18342e7c79f9a5ffcf70ab1123a104e7324c0f2373d823ccb8b"} Jul 06 18:20:19.271830 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.271544 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"3afdb5584e31110bc99a3815f7ce5ba7b58d38773023276ddfffc8f56b32f790"} Jul 06 18:20:19.271830 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.271568 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"6651c34f93a81320bd1413e0f53da54cbd7cb230ac73667094687902dc095c80"} Jul 06 18:20:19.271830 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.271591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"fa45b590f641eb444b477ab97e1be9d5150e4ef0307d3d75c4fc3755285b3bc0"} Jul 06 18:20:19.271830 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.271616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c9b71495-bba2-4605-a416-1f886b718fc6","Type":"ContainerStarted","Data":"4855b63bc744e0f727ec6e79779d8842006ee0bd073c5396d9f0a92a4356c7ab"} Jul 06 18:20:19.302472 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.302140 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.285359394 podStartE2EDuration="2.302119548s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.760154ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:19.299548463 +0000 UTC m=+150.446089410" watchObservedRunningTime="2026-07-06 18:20:19.302119548 +0000 UTC m=+150.448660470" Jul 06 18:20:19.739420 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.739386 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c8c949d8d-r4cxc"] Jul 06 18:20:19.742067 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.742040 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.744509 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.744481 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 18:20:19.744638 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.744505 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 18:20:19.744700 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.744656 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-25b98\"" Jul 06 18:20:19.744771 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.744750 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 18:20:19.744829 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.744780 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 18:20:19.744958 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.744939 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 18:20:19.749688 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.749659 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 18:20:19.760627 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.760604 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c8c949d8d-r4cxc"] Jul 06 18:20:19.808893 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.808862 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-secret-telemeter-client\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.808904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.808925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-metrics-client-ca\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.809011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvzlr\" (UniqueName: \"kubernetes.io/projected/1569f168-17f2-401a-b032-5dac65c6e8ce-kube-api-access-tvzlr\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809061 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.809048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809219 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.809077 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-serving-certs-ca-bundle\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809219 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.809098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-federate-client-tls\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.809219 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.809175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-telemeter-client-tls\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.909822 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-metrics-client-ca\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.909822 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvzlr\" (UniqueName: \"kubernetes.io/projected/1569f168-17f2-401a-b032-5dac65c6e8ce-kube-api-access-tvzlr\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910041 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910041 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-serving-certs-ca-bundle\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910041 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-federate-client-tls\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910041 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-telemeter-client-tls\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910041 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.909960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-secret-telemeter-client\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910280 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.910224 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910728 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.910700 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-metrics-client-ca\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.910839 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.910720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-serving-certs-ca-bundle\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.911125 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.911095 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1569f168-17f2-401a-b032-5dac65c6e8ce-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.913454 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.912658 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.913454 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.912803 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-federate-client-tls\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.913454 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.913088 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-secret-telemeter-client\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.913454 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.913406 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1569f168-17f2-401a-b032-5dac65c6e8ce-telemeter-client-tls\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:19.922502 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:19.922471 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvzlr\" (UniqueName: \"kubernetes.io/projected/1569f168-17f2-401a-b032-5dac65c6e8ce-kube-api-access-tvzlr\") pod \"telemeter-client-c8c949d8d-r4cxc\" (UID: \"1569f168-17f2-401a-b032-5dac65c6e8ce\") " pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:20.043750 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.043653 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:20:20.044072 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.044003 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy" containerID="cri-o://2e4da991916f01e7766d9a25a16a6ef5240de8fc3e63d8e051215898784bf9d1" gracePeriod=600 Jul 06 18:20:20.044072 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.044003 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b8b65c60e0ba7decdab36411b7d5367224690e73c3ee466a1c0893118622ea67" gracePeriod=600 Jul 06 18:20:20.044296 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.044026 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="thanos-sidecar" containerID="cri-o://93212c8ddb7a1f888f1325f725a208f0476bab64a931bfcbd11455a429c3ec93" gracePeriod=600 Jul 06 18:20:20.044296 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.044149 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-web" containerID="cri-o://82ba32968c6cf09a32bbb472e1dff6a39cf0df40dc8127a936e460f0bd783f45" gracePeriod=600 Jul 06 18:20:20.044296 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.044025 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="config-reloader" containerID="cri-o://b5957533836b4c2b2cf216ed566543c784b3f6c93eb58f6b062f23a4ea03964e" gracePeriod=600 Jul 06 18:20:20.044296 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.044057 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="prometheus" containerID="cri-o://9edd2250e64de441f244f15fb10aae321a2f18e0a348ccc55111510715a12f62" gracePeriod=600 Jul 06 18:20:20.055363 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.055128 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" Jul 06 18:20:20.223692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.223660 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c8c949d8d-r4cxc"] Jul 06 18:20:20.226966 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:20:20.226934 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1569f168_17f2_401a_b032_5dac65c6e8ce.slice/crio-be8dc1cab8158d28ebf78b1c51807d7da1e6dfaacbe71ff2f86baa5e772c2604 WatchSource:0}: Error finding container be8dc1cab8158d28ebf78b1c51807d7da1e6dfaacbe71ff2f86baa5e772c2604: Status 404 returned error can't find the container with id be8dc1cab8158d28ebf78b1c51807d7da1e6dfaacbe71ff2f86baa5e772c2604 Jul 06 18:20:20.280865 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280825 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="b8b65c60e0ba7decdab36411b7d5367224690e73c3ee466a1c0893118622ea67" exitCode=0 Jul 06 18:20:20.280865 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280862 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="2e4da991916f01e7766d9a25a16a6ef5240de8fc3e63d8e051215898784bf9d1" exitCode=0 Jul 06 18:20:20.280865 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280874 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="82ba32968c6cf09a32bbb472e1dff6a39cf0df40dc8127a936e460f0bd783f45" exitCode=0 Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280883 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="93212c8ddb7a1f888f1325f725a208f0476bab64a931bfcbd11455a429c3ec93" exitCode=0 Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280891 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="b5957533836b4c2b2cf216ed566543c784b3f6c93eb58f6b062f23a4ea03964e" exitCode=0 Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280898 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerID="9edd2250e64de441f244f15fb10aae321a2f18e0a348ccc55111510715a12f62" exitCode=0 Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.280891 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"b8b65c60e0ba7decdab36411b7d5367224690e73c3ee466a1c0893118622ea67"} Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.281018 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"2e4da991916f01e7766d9a25a16a6ef5240de8fc3e63d8e051215898784bf9d1"} Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.281056 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"82ba32968c6cf09a32bbb472e1dff6a39cf0df40dc8127a936e460f0bd783f45"} Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.281090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"93212c8ddb7a1f888f1325f725a208f0476bab64a931bfcbd11455a429c3ec93"} Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.281124 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"b5957533836b4c2b2cf216ed566543c784b3f6c93eb58f6b062f23a4ea03964e"} Jul 06 18:20:20.281385 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.281155 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"9edd2250e64de441f244f15fb10aae321a2f18e0a348ccc55111510715a12f62"} Jul 06 18:20:20.284199 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.283650 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" event={"ID":"1569f168-17f2-401a-b032-5dac65c6e8ce","Type":"ContainerStarted","Data":"be8dc1cab8158d28ebf78b1c51807d7da1e6dfaacbe71ff2f86baa5e772c2604"} Jul 06 18:20:20.308805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.308782 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:20.415115 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415086 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-tls\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415115 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-web-config\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415141 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-tls-assets\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415171 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-serving-certs-ca-bundle\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415195 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415232 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-metrics-client-certs\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415270 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-kube-rbac-proxy\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415298 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-db\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415347 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-1\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415374 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-0\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415399 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415453 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-thanos-prometheus-http-client-file\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415494 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnqxb\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-kube-api-access-pnqxb\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415519 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-grpc-tls\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415545 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415556 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-metrics-client-ca\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415589 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-2\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415622 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-trusted-ca-bundle\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415652 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config-out\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415686 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-kubelet-serving-ca-bundle\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.415767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415718 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\" (UID: \"0a2f5426-636c-4e9c-a2d2-2a362a427f85\") " Jul 06 18:20:20.416374 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.415989 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.416597 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.416568 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.416657 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.416575 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.417332 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.417124 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.417782 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.417535 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.417782 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.417735 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:20:20.417950 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.417823 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:20:20.417950 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.417903 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.418362 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.418223 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:20:20.418362 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.418309 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.418837 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.418797 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config" (OuterVolumeSpecName: "config") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.419084 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.419048 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.419827 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.419801 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.419827 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.419818 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config-out" (OuterVolumeSpecName: "config-out") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:20:20.419981 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.419851 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.420366 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.420337 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.420447 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.420427 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.420748 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.420720 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.421188 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.421169 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-kube-api-access-pnqxb" (OuterVolumeSpecName: "kube-api-access-pnqxb") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "kube-api-access-pnqxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:20:20.429111 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.429091 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-web-config" (OuterVolumeSpecName: "web-config") pod "0a2f5426-636c-4e9c-a2d2-2a362a427f85" (UID: "0a2f5426-636c-4e9c-a2d2-2a362a427f85"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:20:20.516769 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516722 2566 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.516769 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516757 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnqxb\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-kube-api-access-pnqxb\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.516769 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516773 2566 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-grpc-tls\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516785 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-metrics-client-ca\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516797 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516808 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516820 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config-out\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516831 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516843 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516858 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516869 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-web-config\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516879 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2f5426-636c-4e9c-a2d2-2a362a427f85-tls-assets\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516890 2566 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-config\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516901 2566 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-metrics-client-certs\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516913 2566 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-kube-rbac-proxy\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516924 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-db\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516936 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516949 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a2f5426-636c-4e9c-a2d2-2a362a427f85-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:20.517049 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:20.516963 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a2f5426-636c-4e9c-a2d2-2a362a427f85-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:20:21.289850 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.289824 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.290242 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.289821 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a2f5426-636c-4e9c-a2d2-2a362a427f85","Type":"ContainerDied","Data":"b9acac467525301bc7fb5f5cdbd6466be217c4b0016c2455caa85ecc3f884b2e"} Jul 06 18:20:21.290242 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.289975 2566 scope.go:117] "RemoveContainer" containerID="b8b65c60e0ba7decdab36411b7d5367224690e73c3ee466a1c0893118622ea67" Jul 06 18:20:21.298645 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.298622 2566 scope.go:117] "RemoveContainer" containerID="2e4da991916f01e7766d9a25a16a6ef5240de8fc3e63d8e051215898784bf9d1" Jul 06 18:20:21.306137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.306115 2566 scope.go:117] "RemoveContainer" containerID="82ba32968c6cf09a32bbb472e1dff6a39cf0df40dc8127a936e460f0bd783f45" Jul 06 18:20:21.312931 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.312912 2566 scope.go:117] "RemoveContainer" containerID="93212c8ddb7a1f888f1325f725a208f0476bab64a931bfcbd11455a429c3ec93" Jul 06 18:20:21.314525 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.314297 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:20:21.318889 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.318867 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:20:21.320735 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.320719 2566 scope.go:117] "RemoveContainer" containerID="b5957533836b4c2b2cf216ed566543c784b3f6c93eb58f6b062f23a4ea03964e" Jul 06 18:20:21.328750 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.328730 2566 scope.go:117] "RemoveContainer" containerID="9edd2250e64de441f244f15fb10aae321a2f18e0a348ccc55111510715a12f62" Jul 06 18:20:21.337485 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.337465 2566 scope.go:117] "RemoveContainer" containerID="1f6e7a212c447a52aca00b1b643f78b1dfdb5ef28623ebc489d0526ff5502e94" Jul 06 18:20:21.346767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.346740 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:20:21.347207 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347186 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="prometheus" Jul 06 18:20:21.347207 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347207 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="prometheus" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347221 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="thanos-sidecar" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347226 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="thanos-sidecar" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347238 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="init-config-reloader" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347244 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="init-config-reloader" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347257 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-thanos" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347262 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-thanos" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347270 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="config-reloader" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347274 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="config-reloader" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347281 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-web" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347287 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-web" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347296 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy" Jul 06 18:20:21.347407 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347304 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy" Jul 06 18:20:21.347851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347442 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="thanos-sidecar" Jul 06 18:20:21.347851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347484 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="config-reloader" Jul 06 18:20:21.347851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347491 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy" Jul 06 18:20:21.347851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347519 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-web" Jul 06 18:20:21.347851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347542 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="prometheus" Jul 06 18:20:21.347851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.347551 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" containerName="kube-rbac-proxy-thanos" Jul 06 18:20:21.351522 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.351496 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.354814 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.355067 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.355234 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pbmn4\"" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.355411 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.355607 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.355790 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 18:20:21.356055 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.355930 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 18:20:21.356457 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356094 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4teih7ntrasis\"" Jul 06 18:20:21.356457 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356282 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 18:20:21.356561 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356462 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 18:20:21.356561 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356510 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 18:20:21.356657 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356641 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 18:20:21.356748 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356719 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 18:20:21.356831 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.356751 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 18:20:21.358998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.358861 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 18:20:21.361551 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.361532 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 18:20:21.368926 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.368896 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423739 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h88jj\" (UniqueName: \"kubernetes.io/projected/eaa9a54c-39df-4941-90c8-d74f8301e887-kube-api-access-h88jj\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423921 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.423999 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424065 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaa9a54c-39df-4941-90c8-d74f8301e887-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424090 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.424595 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424256 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-config\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.425401 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424282 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaa9a54c-39df-4941-90c8-d74f8301e887-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.425401 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.425401 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.425401 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.424376 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525305 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaa9a54c-39df-4941-90c8-d74f8301e887-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525305 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525400 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525427 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525521 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h88jj\" (UniqueName: \"kubernetes.io/projected/eaa9a54c-39df-4941-90c8-d74f8301e887-kube-api-access-h88jj\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.525591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525572 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaa9a54c-39df-4941-90c8-d74f8301e887-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525836 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526075 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.525847 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-config\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526543 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.526215 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.526543 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.526452 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.528020 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.526996 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.528020 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.527672 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.528201 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.528016 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.528386 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.528363 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaa9a54c-39df-4941-90c8-d74f8301e887-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.528850 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.528823 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.530214 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.529019 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-config\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.530214 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.529415 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.530214 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.529428 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.530214 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.530095 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.530630 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.530604 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.530751 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.530727 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaa9a54c-39df-4941-90c8-d74f8301e887-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.531027 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.531002 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.531553 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.531528 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.531673 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.531643 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.532597 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.532577 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaa9a54c-39df-4941-90c8-d74f8301e887-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.532655 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.532586 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaa9a54c-39df-4941-90c8-d74f8301e887-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.533168 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.533148 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a2f5426-636c-4e9c-a2d2-2a362a427f85" path="/var/lib/kubelet/pods/0a2f5426-636c-4e9c-a2d2-2a362a427f85/volumes" Jul 06 18:20:21.536870 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.536850 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h88jj\" (UniqueName: \"kubernetes.io/projected/eaa9a54c-39df-4941-90c8-d74f8301e887-kube-api-access-h88jj\") pod \"prometheus-k8s-0\" (UID: \"eaa9a54c-39df-4941-90c8-d74f8301e887\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.666056 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.665957 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:20:21.809529 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:21.809466 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:20:21.811778 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:20:21.811751 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa9a54c_39df_4941_90c8_d74f8301e887.slice/crio-284c94751792486c5dc93ea58311ddd7c6179ee248d64ebc5437f9b5c1270c6f WatchSource:0}: Error finding container 284c94751792486c5dc93ea58311ddd7c6179ee248d64ebc5437f9b5c1270c6f: Status 404 returned error can't find the container with id 284c94751792486c5dc93ea58311ddd7c6179ee248d64ebc5437f9b5c1270c6f Jul 06 18:20:22.297495 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:22.297457 2566 generic.go:356] "Generic (PLEG): container finished" podID="eaa9a54c-39df-4941-90c8-d74f8301e887" containerID="8e2caacff9caa2a83410b797cfd9d468b103bd51b8eb59b1b0442f5eb0ada865" exitCode=0 Jul 06 18:20:22.297952 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:22.297551 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerDied","Data":"8e2caacff9caa2a83410b797cfd9d468b103bd51b8eb59b1b0442f5eb0ada865"} Jul 06 18:20:22.297952 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:22.297647 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"284c94751792486c5dc93ea58311ddd7c6179ee248d64ebc5437f9b5c1270c6f"} Jul 06 18:20:23.304972 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.304927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" event={"ID":"1569f168-17f2-401a-b032-5dac65c6e8ce","Type":"ContainerStarted","Data":"783086ad37d8e15942cfdc6cddcf42b317aad22974a84c31f301a9f7a7caced4"} Jul 06 18:20:23.305409 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.305007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" event={"ID":"1569f168-17f2-401a-b032-5dac65c6e8ce","Type":"ContainerStarted","Data":"70cbd1c31abb698b82ba1b9768984aa4a40fc580b7e67fdd2bfea41ee36dfc94"} Jul 06 18:20:23.305409 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.305038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" event={"ID":"1569f168-17f2-401a-b032-5dac65c6e8ce","Type":"ContainerStarted","Data":"3b22497520961248296b77ea6ee1963eb7ded7c87a5354a22bd012d09a90d699"} Jul 06 18:20:23.308131 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.308106 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"4ba9155fd7a1983aeefda900e0fbd2250695919ceb13235312057ca318f1e58c"} Jul 06 18:20:23.308252 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.308173 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"7b0ca2366f689194384cfe367f0191f25d8eecd88522502517e8289d047de4f0"} Jul 06 18:20:23.308252 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.308214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"e166b5b020992a4330917d1444ff643f3fd007b3a4aaa8b18a5c0fddeaea496e"} Jul 06 18:20:23.308339 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.308254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"e6c3fc1d66561f73275fc08a77b9fd994166bc1d9849fa0bc700b650daefb7cf"} Jul 06 18:20:23.308339 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.308282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"9f5fbde2d30d9d4bcc94ab966edf9c648b92c047efd0ee045e364eaf98f0ca11"} Jul 06 18:20:23.308339 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.308306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa9a54c-39df-4941-90c8-d74f8301e887","Type":"ContainerStarted","Data":"2cb7f19b51c22a05319d9b06f9ac20073d56edd42f5607c3a78932ed265a7799"} Jul 06 18:20:23.327207 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.327162 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c8c949d8d-r4cxc" podStartSLOduration=2.033558896 podStartE2EDuration="4.327148014s" totalImagesPullingTime="2.293589118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:23.32488205 +0000 UTC m=+154.471422974" watchObservedRunningTime="2026-07-06 18:20:23.327148014 +0000 UTC m=+154.473688938" Jul 06 18:20:23.353248 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:23.353201 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.327666641 podStartE2EDuration="2.353187887s" totalImagesPullingTime="0s" totalInitContainerRuntime="25.521246ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:23.352560457 +0000 UTC m=+154.499101392" watchObservedRunningTime="2026-07-06 18:20:23.353187887 +0000 UTC m=+154.499728808" Jul 06 18:20:26.666577 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:20:26.666536 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:21:21.666175 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:21:21.666137 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:21:21.681555 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:21:21.681533 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:21:22.556200 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:21:22.556173 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:22:49.245151 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:22:49.245123 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 18:23:14.047522 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.047486 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zzxnz"] Jul 06 18:23:14.049810 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.049790 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.053129 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.053109 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-m7ht8\"" Jul 06 18:23:14.053368 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.053353 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 18:23:14.054485 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.054466 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 18:23:14.062005 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.061983 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zzxnz"] Jul 06 18:23:14.163929 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.163886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a4076c4-b059-4037-b7ef-32fa2a92c842-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zzxnz\" (UID: \"5a4076c4-b059-4037-b7ef-32fa2a92c842\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.163929 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.163937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxz5l\" (UniqueName: \"kubernetes.io/projected/5a4076c4-b059-4037-b7ef-32fa2a92c842-kube-api-access-bxz5l\") pod \"cert-manager-webhook-6486ffd997-zzxnz\" (UID: \"5a4076c4-b059-4037-b7ef-32fa2a92c842\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.264791 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.264754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a4076c4-b059-4037-b7ef-32fa2a92c842-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zzxnz\" (UID: \"5a4076c4-b059-4037-b7ef-32fa2a92c842\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.264791 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.264796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxz5l\" (UniqueName: \"kubernetes.io/projected/5a4076c4-b059-4037-b7ef-32fa2a92c842-kube-api-access-bxz5l\") pod \"cert-manager-webhook-6486ffd997-zzxnz\" (UID: \"5a4076c4-b059-4037-b7ef-32fa2a92c842\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.281431 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.281398 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a4076c4-b059-4037-b7ef-32fa2a92c842-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zzxnz\" (UID: \"5a4076c4-b059-4037-b7ef-32fa2a92c842\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.281603 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.281577 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxz5l\" (UniqueName: \"kubernetes.io/projected/5a4076c4-b059-4037-b7ef-32fa2a92c842-kube-api-access-bxz5l\") pod \"cert-manager-webhook-6486ffd997-zzxnz\" (UID: \"5a4076c4-b059-4037-b7ef-32fa2a92c842\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.373213 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.373121 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:14.505004 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.504968 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zzxnz"] Jul 06 18:23:14.508199 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:23:14.508173 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a4076c4_b059_4037_b7ef_32fa2a92c842.slice/crio-1d11974821ea7a8f7ac535cb05a571901c6dba84ea0abf1d5729b3ad66abd308 WatchSource:0}: Error finding container 1d11974821ea7a8f7ac535cb05a571901c6dba84ea0abf1d5729b3ad66abd308: Status 404 returned error can't find the container with id 1d11974821ea7a8f7ac535cb05a571901c6dba84ea0abf1d5729b3ad66abd308 Jul 06 18:23:14.510089 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.510071 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:23:14.993955 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:14.993923 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" event={"ID":"5a4076c4-b059-4037-b7ef-32fa2a92c842","Type":"ContainerStarted","Data":"1d11974821ea7a8f7ac535cb05a571901c6dba84ea0abf1d5729b3ad66abd308"} Jul 06 18:23:19.016171 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:19.016134 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" event={"ID":"5a4076c4-b059-4037-b7ef-32fa2a92c842","Type":"ContainerStarted","Data":"894f2caaa02f9b924b5330631214fe3334e1b14d0879d2de418001312dc8f750"} Jul 06 18:23:19.016658 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:19.016285 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:19.033879 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:19.033832 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" podStartSLOduration=1.581977778 podStartE2EDuration="5.033817088s" totalImagesPullingTime="3.45183931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:19.032123234 +0000 UTC m=+330.178664159" watchObservedRunningTime="2026-07-06 18:23:19.033817088 +0000 UTC m=+330.180358009" Jul 06 18:23:25.022110 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:25.022082 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zzxnz" Jul 06 18:23:28.906732 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.906695 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb"] Jul 06 18:23:28.909027 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.909008 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:28.911473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.911452 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 18:23:28.912489 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.912470 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:23:28.912581 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.912473 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rhg5c\"" Jul 06 18:23:28.920452 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.920431 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb"] Jul 06 18:23:28.994448 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.994412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7865cd18-4584-4c20-b3e2-6081251e5269-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qngqb\" (UID: \"7865cd18-4584-4c20-b3e2-6081251e5269\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:28.994639 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:28.994582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9cpp\" (UniqueName: \"kubernetes.io/projected/7865cd18-4584-4c20-b3e2-6081251e5269-kube-api-access-b9cpp\") pod \"openshift-lws-operator-fd8ccff4c-qngqb\" (UID: \"7865cd18-4584-4c20-b3e2-6081251e5269\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:29.095914 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:29.095883 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7865cd18-4584-4c20-b3e2-6081251e5269-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qngqb\" (UID: \"7865cd18-4584-4c20-b3e2-6081251e5269\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:29.096097 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:29.095988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9cpp\" (UniqueName: \"kubernetes.io/projected/7865cd18-4584-4c20-b3e2-6081251e5269-kube-api-access-b9cpp\") pod \"openshift-lws-operator-fd8ccff4c-qngqb\" (UID: \"7865cd18-4584-4c20-b3e2-6081251e5269\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:29.096427 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:29.096407 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7865cd18-4584-4c20-b3e2-6081251e5269-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qngqb\" (UID: \"7865cd18-4584-4c20-b3e2-6081251e5269\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:29.104670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:29.104643 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9cpp\" (UniqueName: \"kubernetes.io/projected/7865cd18-4584-4c20-b3e2-6081251e5269-kube-api-access-b9cpp\") pod \"openshift-lws-operator-fd8ccff4c-qngqb\" (UID: \"7865cd18-4584-4c20-b3e2-6081251e5269\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:29.218376 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:29.218345 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" Jul 06 18:23:29.345469 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:29.345443 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb"] Jul 06 18:23:29.347667 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:23:29.347642 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7865cd18_4584_4c20_b3e2_6081251e5269.slice/crio-6735e5ab89da1da692d91967f7d282325e1dff058cf5f105ca1e4f2a5f272ec7 WatchSource:0}: Error finding container 6735e5ab89da1da692d91967f7d282325e1dff058cf5f105ca1e4f2a5f272ec7: Status 404 returned error can't find the container with id 6735e5ab89da1da692d91967f7d282325e1dff058cf5f105ca1e4f2a5f272ec7 Jul 06 18:23:30.062380 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:30.062348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" event={"ID":"7865cd18-4584-4c20-b3e2-6081251e5269","Type":"ContainerStarted","Data":"6735e5ab89da1da692d91967f7d282325e1dff058cf5f105ca1e4f2a5f272ec7"} Jul 06 18:23:33.079718 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:33.079675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" event={"ID":"7865cd18-4584-4c20-b3e2-6081251e5269","Type":"ContainerStarted","Data":"9586eee569271334542397cdb2d63a4b2596aa07ee812f8b43224ebdbeb4233b"} Jul 06 18:23:33.095976 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:33.095925 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qngqb" podStartSLOduration=2.43853727 podStartE2EDuration="5.095909981s" totalImagesPullingTime="2.657372711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:33.094659179 +0000 UTC m=+344.241200114" watchObservedRunningTime="2026-07-06 18:23:33.095909981 +0000 UTC m=+344.242450902" Jul 06 18:23:50.237718 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.237638 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp"] Jul 06 18:23:50.244556 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.244531 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.248878 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.248856 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v5sm7\"" Jul 06 18:23:50.249841 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.249819 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 18:23:50.250143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.249929 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 18:23:50.250618 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.250600 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 18:23:50.252172 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.252149 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp"] Jul 06 18:23:50.293812 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.293776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dc1501ca-b812-42aa-9433-053ba95dd289-manager-config\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.293968 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.293843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7t8w\" (UniqueName: \"kubernetes.io/projected/dc1501ca-b812-42aa-9433-053ba95dd289-kube-api-access-c7t8w\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.293968 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.293871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc1501ca-b812-42aa-9433-053ba95dd289-metrics-cert\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.293968 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.293898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc1501ca-b812-42aa-9433-053ba95dd289-cert\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.395291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.395257 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7t8w\" (UniqueName: \"kubernetes.io/projected/dc1501ca-b812-42aa-9433-053ba95dd289-kube-api-access-c7t8w\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.395291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.395293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc1501ca-b812-42aa-9433-053ba95dd289-metrics-cert\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.395530 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.395503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc1501ca-b812-42aa-9433-053ba95dd289-cert\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.395670 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.395653 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dc1501ca-b812-42aa-9433-053ba95dd289-manager-config\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.396263 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.396245 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dc1501ca-b812-42aa-9433-053ba95dd289-manager-config\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.397801 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.397785 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc1501ca-b812-42aa-9433-053ba95dd289-metrics-cert\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.397865 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.397801 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc1501ca-b812-42aa-9433-053ba95dd289-cert\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.408519 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.408495 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7t8w\" (UniqueName: \"kubernetes.io/projected/dc1501ca-b812-42aa-9433-053ba95dd289-kube-api-access-c7t8w\") pod \"lws-controller-manager-c45f78566-mbnmp\" (UID: \"dc1501ca-b812-42aa-9433-053ba95dd289\") " pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.556606 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.556525 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:23:50.697687 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:50.697656 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp"] Jul 06 18:23:50.700363 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:23:50.700333 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1501ca_b812_42aa_9433_053ba95dd289.slice/crio-b36cd1d3b1851cfb5613a767fd74bd69e8fd49c7c15cb7ca7059d5842634584f WatchSource:0}: Error finding container b36cd1d3b1851cfb5613a767fd74bd69e8fd49c7c15cb7ca7059d5842634584f: Status 404 returned error can't find the container with id b36cd1d3b1851cfb5613a767fd74bd69e8fd49c7c15cb7ca7059d5842634584f Jul 06 18:23:51.155008 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.154975 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" event={"ID":"dc1501ca-b812-42aa-9433-053ba95dd289","Type":"ContainerStarted","Data":"b36cd1d3b1851cfb5613a767fd74bd69e8fd49c7c15cb7ca7059d5842634584f"} Jul 06 18:23:51.304947 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.304910 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4"] Jul 06 18:23:51.360658 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.360624 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4"] Jul 06 18:23:51.360807 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.360743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.365157 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.365135 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 18:23:51.365299 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.365179 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 18:23:51.365591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.365568 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 18:23:51.365727 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.365596 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 18:23:51.365727 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.365573 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mhwpf\"" Jul 06 18:23:51.404673 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.404644 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c5acd9d-4795-43be-842e-9df64881ddd3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.404814 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.404698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c5acd9d-4795-43be-842e-9df64881ddd3-tmp\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.404814 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.404744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8s7j\" (UniqueName: \"kubernetes.io/projected/9c5acd9d-4795-43be-842e-9df64881ddd3-kube-api-access-c8s7j\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.404814 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.404801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c5acd9d-4795-43be-842e-9df64881ddd3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.404919 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.404858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c5acd9d-4795-43be-842e-9df64881ddd3-manifests\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.506224 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.506185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c5acd9d-4795-43be-842e-9df64881ddd3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.506430 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.506252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c5acd9d-4795-43be-842e-9df64881ddd3-tmp\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.506430 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.506280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8s7j\" (UniqueName: \"kubernetes.io/projected/9c5acd9d-4795-43be-842e-9df64881ddd3-kube-api-access-c8s7j\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.506430 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.506297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c5acd9d-4795-43be-842e-9df64881ddd3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.506430 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.506364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c5acd9d-4795-43be-842e-9df64881ddd3-manifests\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.506846 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.506817 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c5acd9d-4795-43be-842e-9df64881ddd3-manifests\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.508717 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.508696 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c5acd9d-4795-43be-842e-9df64881ddd3-tmp\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.508924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.508909 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c5acd9d-4795-43be-842e-9df64881ddd3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.508966 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.508921 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c5acd9d-4795-43be-842e-9df64881ddd3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.517378 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.517350 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8s7j\" (UniqueName: \"kubernetes.io/projected/9c5acd9d-4795-43be-842e-9df64881ddd3-kube-api-access-c8s7j\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-ckrl4\" (UID: \"9c5acd9d-4795-43be-842e-9df64881ddd3\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.671707 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.671675 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:51.835722 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:51.835566 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4"] Jul 06 18:23:51.838614 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:23:51.838589 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5acd9d_4795_43be_842e_9df64881ddd3.slice/crio-0412a4220135fa02fc0f4e81aa1813f93f6a63868d234bdcf30532bf75a71e24 WatchSource:0}: Error finding container 0412a4220135fa02fc0f4e81aa1813f93f6a63868d234bdcf30532bf75a71e24: Status 404 returned error can't find the container with id 0412a4220135fa02fc0f4e81aa1813f93f6a63868d234bdcf30532bf75a71e24 Jul 06 18:23:52.161164 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:52.161079 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" event={"ID":"9c5acd9d-4795-43be-842e-9df64881ddd3","Type":"ContainerStarted","Data":"0412a4220135fa02fc0f4e81aa1813f93f6a63868d234bdcf30532bf75a71e24"} Jul 06 18:23:57.189935 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:57.189897 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" event={"ID":"9c5acd9d-4795-43be-842e-9df64881ddd3","Type":"ContainerStarted","Data":"1c3ac8978e59b498e29e8c57b26f5095a3a657504f9168d4a0b18732d884778a"} Jul 06 18:23:58.195579 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:58.195537 2566 generic.go:356] "Generic (PLEG): container finished" podID="9c5acd9d-4795-43be-842e-9df64881ddd3" containerID="1c3ac8978e59b498e29e8c57b26f5095a3a657504f9168d4a0b18732d884778a" exitCode=0 Jul 06 18:23:58.195579 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:58.195582 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" event={"ID":"9c5acd9d-4795-43be-842e-9df64881ddd3","Type":"ContainerDied","Data":"1c3ac8978e59b498e29e8c57b26f5095a3a657504f9168d4a0b18732d884778a"} Jul 06 18:23:59.201599 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:59.201566 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" event={"ID":"9c5acd9d-4795-43be-842e-9df64881ddd3","Type":"ContainerStarted","Data":"3cc60d05b9c2391d5d04400fbf28350c3faf69a601552f70171cf55be5c38333"} Jul 06 18:23:59.202008 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:59.201672 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:23:59.224620 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:23:59.224567 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" podStartSLOduration=2.192442978 podStartE2EDuration="8.224552596s" totalImagesPullingTime="4.982630408s" totalInitContainerRuntime="1.04947921s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:59.223202688 +0000 UTC m=+370.369743620" watchObservedRunningTime="2026-07-06 18:23:59.224552596 +0000 UTC m=+370.371093516" Jul 06 18:24:02.216368 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:02.216304 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" event={"ID":"dc1501ca-b812-42aa-9433-053ba95dd289","Type":"ContainerStarted","Data":"f3c6ad5e0d1087716a983e7a2e945233220378c2bb19b485c18946d968afff5d"} Jul 06 18:24:02.216825 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:02.216519 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:24:02.233374 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:02.233306 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" podStartSLOduration=1.407728864 podStartE2EDuration="12.233291301s" totalImagesPullingTime="10.825562437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:02.231986545 +0000 UTC m=+373.378527491" watchObservedRunningTime="2026-07-06 18:24:02.233291301 +0000 UTC m=+373.379832222" Jul 06 18:24:10.207865 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:10.207832 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-ckrl4" Jul 06 18:24:12.995223 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:12.995176 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn"] Jul 06 18:24:12.999396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:12.999373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.003145 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.003119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 18:24:13.003336 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.003158 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6lwvb\"" Jul 06 18:24:13.003336 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.003247 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 18:24:13.004104 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.004085 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 18:24:13.004450 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.004432 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 18:24:13.019826 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.019797 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn"] Jul 06 18:24:13.105723 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.105682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a59f1815-bef4-465d-93c7-e7e804c9f536-tls-certs\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.105723 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.105736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a59f1815-bef4-465d-93c7-e7e804c9f536-tmp\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.105956 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.105752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxz4\" (UniqueName: \"kubernetes.io/projected/a59f1815-bef4-465d-93c7-e7e804c9f536-kube-api-access-wwxz4\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.206796 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.206762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a59f1815-bef4-465d-93c7-e7e804c9f536-tls-certs\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.206796 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.206800 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a59f1815-bef4-465d-93c7-e7e804c9f536-tmp\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.207046 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.206818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxz4\" (UniqueName: \"kubernetes.io/projected/a59f1815-bef4-465d-93c7-e7e804c9f536-kube-api-access-wwxz4\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.209100 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.209073 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a59f1815-bef4-465d-93c7-e7e804c9f536-tmp\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.209290 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.209272 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a59f1815-bef4-465d-93c7-e7e804c9f536-tls-certs\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.215368 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.215307 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxz4\" (UniqueName: \"kubernetes.io/projected/a59f1815-bef4-465d-93c7-e7e804c9f536-kube-api-access-wwxz4\") pod \"kube-auth-proxy-68bf6db468-5pqvn\" (UID: \"a59f1815-bef4-465d-93c7-e7e804c9f536\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.223946 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.223926 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c45f78566-mbnmp" Jul 06 18:24:13.310589 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.310515 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" Jul 06 18:24:13.495767 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:13.495740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn"] Jul 06 18:24:13.497953 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:24:13.497919 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59f1815_bef4_465d_93c7_e7e804c9f536.slice/crio-11ab2c063ff31b9c194760af6348c183ecd83dc081d67805b72445ccd0ddab6d WatchSource:0}: Error finding container 11ab2c063ff31b9c194760af6348c183ecd83dc081d67805b72445ccd0ddab6d: Status 404 returned error can't find the container with id 11ab2c063ff31b9c194760af6348c183ecd83dc081d67805b72445ccd0ddab6d Jul 06 18:24:14.274923 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:14.274878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" event={"ID":"a59f1815-bef4-465d-93c7-e7e804c9f536","Type":"ContainerStarted","Data":"11ab2c063ff31b9c194760af6348c183ecd83dc081d67805b72445ccd0ddab6d"} Jul 06 18:24:17.292948 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:17.292908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" event={"ID":"a59f1815-bef4-465d-93c7-e7e804c9f536","Type":"ContainerStarted","Data":"6bc08609504ce486f5e11fa56f18872f4cb040b0cb95ec40c3a83328867039aa"} Jul 06 18:24:17.311396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:24:17.311343 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68bf6db468-5pqvn" podStartSLOduration=1.8676825419999998 podStartE2EDuration="5.311311725s" totalImagesPullingTime="3.443629183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:17.309728219 +0000 UTC m=+388.456269144" watchObservedRunningTime="2026-07-06 18:24:17.311311725 +0000 UTC m=+388.457852646" Jul 06 18:25:57.765471 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.765440 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s"] Jul 06 18:25:57.769143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.769123 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:57.771449 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.771431 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 18:25:57.776400 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.772217 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 18:25:57.776400 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.772225 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5p2gs\"" Jul 06 18:25:57.776400 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.772875 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 18:25:57.776400 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.772948 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 18:25:57.781636 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.779159 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s"] Jul 06 18:25:57.948994 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.948955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/377ee50b-a710-4a3b-b579-3a80d69cb41f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:57.949166 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.949079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldvsv\" (UniqueName: \"kubernetes.io/projected/377ee50b-a710-4a3b-b579-3a80d69cb41f-kube-api-access-ldvsv\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:57.949166 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:57.949113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/377ee50b-a710-4a3b-b579-3a80d69cb41f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.049863 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.049783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/377ee50b-a710-4a3b-b579-3a80d69cb41f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.050022 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.049873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldvsv\" (UniqueName: \"kubernetes.io/projected/377ee50b-a710-4a3b-b579-3a80d69cb41f-kube-api-access-ldvsv\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.050022 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.049897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/377ee50b-a710-4a3b-b579-3a80d69cb41f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.050534 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.050511 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/377ee50b-a710-4a3b-b579-3a80d69cb41f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.052133 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.052104 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/377ee50b-a710-4a3b-b579-3a80d69cb41f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.086346 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.086288 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldvsv\" (UniqueName: \"kubernetes.io/projected/377ee50b-a710-4a3b-b579-3a80d69cb41f-kube-api-access-ldvsv\") pod \"kuadrant-console-plugin-66b5774985-r7m7s\" (UID: \"377ee50b-a710-4a3b-b579-3a80d69cb41f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.384855 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.384770 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" Jul 06 18:25:58.513942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.513904 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s"] Jul 06 18:25:58.516248 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:25:58.516224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod377ee50b_a710_4a3b_b579_3a80d69cb41f.slice/crio-8b489ecee8c475eddf813e78bf7cdd2d932340da2447ecc83c6925521712cce4 WatchSource:0}: Error finding container 8b489ecee8c475eddf813e78bf7cdd2d932340da2447ecc83c6925521712cce4: Status 404 returned error can't find the container with id 8b489ecee8c475eddf813e78bf7cdd2d932340da2447ecc83c6925521712cce4 Jul 06 18:25:58.751190 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:25:58.751157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" event={"ID":"377ee50b-a710-4a3b-b579-3a80d69cb41f","Type":"ContainerStarted","Data":"8b489ecee8c475eddf813e78bf7cdd2d932340da2447ecc83c6925521712cce4"} Jul 06 18:26:08.267268 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.267229 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649"] Jul 06 18:26:08.270633 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.270616 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.273917 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.273895 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v4csq\"" Jul 06 18:26:08.286520 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.286497 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649"] Jul 06 18:26:08.346668 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.346631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzg49\" (UniqueName: \"kubernetes.io/projected/85de066e-1490-4c28-ae3b-11fd32d7e5f4-kube-api-access-pzg49\") pod \"kuadrant-operator-controller-manager-86878c769f-8k649\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.346925 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.346699 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85de066e-1490-4c28-ae3b-11fd32d7e5f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8k649\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.447591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.447545 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85de066e-1490-4c28-ae3b-11fd32d7e5f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8k649\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.447775 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.447720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzg49\" (UniqueName: \"kubernetes.io/projected/85de066e-1490-4c28-ae3b-11fd32d7e5f4-kube-api-access-pzg49\") pod \"kuadrant-operator-controller-manager-86878c769f-8k649\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.448031 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.448009 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85de066e-1490-4c28-ae3b-11fd32d7e5f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8k649\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.456748 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.456717 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzg49\" (UniqueName: \"kubernetes.io/projected/85de066e-1490-4c28-ae3b-11fd32d7e5f4-kube-api-access-pzg49\") pod \"kuadrant-operator-controller-manager-86878c769f-8k649\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.580780 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.580686 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:08.747239 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:08.747204 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649"] Jul 06 18:26:22.893484 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:26:22.893446 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85de066e_1490_4c28_ae3b_11fd32d7e5f4.slice/crio-67578b5225fcb6636ca62d517ab358c7803b6b8ce5f5fdc095493ce110609606 WatchSource:0}: Error finding container 67578b5225fcb6636ca62d517ab358c7803b6b8ce5f5fdc095493ce110609606: Status 404 returned error can't find the container with id 67578b5225fcb6636ca62d517ab358c7803b6b8ce5f5fdc095493ce110609606 Jul 06 18:26:23.886907 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:23.886860 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" event={"ID":"377ee50b-a710-4a3b-b579-3a80d69cb41f","Type":"ContainerStarted","Data":"297916f3f12061cf8a847aff9dc274c02d62257cec9681e3ac7877eb376960b9"} Jul 06 18:26:23.888584 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:23.888479 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" event={"ID":"85de066e-1490-4c28-ae3b-11fd32d7e5f4","Type":"ContainerStarted","Data":"67578b5225fcb6636ca62d517ab358c7803b6b8ce5f5fdc095493ce110609606"} Jul 06 18:26:23.902920 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:23.902849 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r7m7s" podStartSLOduration=2.445799906 podStartE2EDuration="26.902828548s" totalImagesPullingTime="24.457028642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:23.902437511 +0000 UTC m=+515.048978435" watchObservedRunningTime="2026-07-06 18:26:23.902828548 +0000 UTC m=+515.049369470" Jul 06 18:26:27.916638 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:27.916584 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" event={"ID":"85de066e-1490-4c28-ae3b-11fd32d7e5f4","Type":"ContainerStarted","Data":"055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676"} Jul 06 18:26:27.917101 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:27.916767 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:27.936806 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:27.936759 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" podStartSLOduration=15.843194525 podStartE2EDuration="19.936744639s" totalImagesPullingTime="4.093550114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:27.934533728 +0000 UTC m=+519.081074652" watchObservedRunningTime="2026-07-06 18:26:27.936744639 +0000 UTC m=+519.083285561" Jul 06 18:26:38.923626 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:38.923596 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:26:58.582207 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.582127 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-6fjml"] Jul 06 18:26:58.613979 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.613950 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6fjml"] Jul 06 18:26:58.614132 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.614045 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:26:58.616987 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.616965 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8zw9t\"" Jul 06 18:26:58.725817 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.725782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mpv2\" (UniqueName: \"kubernetes.io/projected/e45306e8-629d-4915-8598-16c1f3fea49d-kube-api-access-2mpv2\") pod \"authorino-75d556bd77-6fjml\" (UID: \"e45306e8-629d-4915-8598-16c1f3fea49d\") " pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:26:58.826750 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.826716 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mpv2\" (UniqueName: \"kubernetes.io/projected/e45306e8-629d-4915-8598-16c1f3fea49d-kube-api-access-2mpv2\") pod \"authorino-75d556bd77-6fjml\" (UID: \"e45306e8-629d-4915-8598-16c1f3fea49d\") " pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:26:58.834600 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.834552 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mpv2\" (UniqueName: \"kubernetes.io/projected/e45306e8-629d-4915-8598-16c1f3fea49d-kube-api-access-2mpv2\") pod \"authorino-75d556bd77-6fjml\" (UID: \"e45306e8-629d-4915-8598-16c1f3fea49d\") " pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:26:58.923474 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:58.923445 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:26:59.048758 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:59.048728 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6fjml"] Jul 06 18:26:59.052152 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:26:59.052122 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode45306e8_629d_4915_8598_16c1f3fea49d.slice/crio-a4c7c00b63926d65fe109d6c637cf0061c9a51bb6d205b9ae3cf4d8947b6cf32 WatchSource:0}: Error finding container a4c7c00b63926d65fe109d6c637cf0061c9a51bb6d205b9ae3cf4d8947b6cf32: Status 404 returned error can't find the container with id a4c7c00b63926d65fe109d6c637cf0061c9a51bb6d205b9ae3cf4d8947b6cf32 Jul 06 18:26:59.064368 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:26:59.064339 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6fjml" event={"ID":"e45306e8-629d-4915-8598-16c1f3fea49d","Type":"ContainerStarted","Data":"a4c7c00b63926d65fe109d6c637cf0061c9a51bb6d205b9ae3cf4d8947b6cf32"} Jul 06 18:27:03.087267 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:03.087170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6fjml" event={"ID":"e45306e8-629d-4915-8598-16c1f3fea49d","Type":"ContainerStarted","Data":"a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab"} Jul 06 18:27:03.110463 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:03.110412 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-6fjml" podStartSLOduration=1.334213495 podStartE2EDuration="5.110375293s" totalImagesPullingTime="3.776161798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:03.109159313 +0000 UTC m=+554.255700231" watchObservedRunningTime="2026-07-06 18:27:03.110375293 +0000 UTC m=+554.256916235" Jul 06 18:27:28.740409 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:28.740373 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6fjml"] Jul 06 18:27:28.740936 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:28.740561 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-6fjml" podUID="e45306e8-629d-4915-8598-16c1f3fea49d" containerName="authorino" containerID="cri-o://a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab" gracePeriod=30 Jul 06 18:27:28.990650 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:28.990592 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:27:29.002104 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.002083 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mpv2\" (UniqueName: \"kubernetes.io/projected/e45306e8-629d-4915-8598-16c1f3fea49d-kube-api-access-2mpv2\") pod \"e45306e8-629d-4915-8598-16c1f3fea49d\" (UID: \"e45306e8-629d-4915-8598-16c1f3fea49d\") " Jul 06 18:27:29.004347 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.004300 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45306e8-629d-4915-8598-16c1f3fea49d-kube-api-access-2mpv2" (OuterVolumeSpecName: "kube-api-access-2mpv2") pod "e45306e8-629d-4915-8598-16c1f3fea49d" (UID: "e45306e8-629d-4915-8598-16c1f3fea49d"). InnerVolumeSpecName "kube-api-access-2mpv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:27:29.103001 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.102970 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mpv2\" (UniqueName: \"kubernetes.io/projected/e45306e8-629d-4915-8598-16c1f3fea49d-kube-api-access-2mpv2\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:27:29.206287 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.206258 2566 generic.go:356] "Generic (PLEG): container finished" podID="e45306e8-629d-4915-8598-16c1f3fea49d" containerID="a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab" exitCode=0 Jul 06 18:27:29.206458 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.206300 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6fjml" Jul 06 18:27:29.206458 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.206345 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6fjml" event={"ID":"e45306e8-629d-4915-8598-16c1f3fea49d","Type":"ContainerDied","Data":"a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab"} Jul 06 18:27:29.206458 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.206385 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6fjml" event={"ID":"e45306e8-629d-4915-8598-16c1f3fea49d","Type":"ContainerDied","Data":"a4c7c00b63926d65fe109d6c637cf0061c9a51bb6d205b9ae3cf4d8947b6cf32"} Jul 06 18:27:29.206458 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.206402 2566 scope.go:117] "RemoveContainer" containerID="a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab" Jul 06 18:27:29.215132 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.215117 2566 scope.go:117] "RemoveContainer" containerID="a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab" Jul 06 18:27:29.215413 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:27:29.215393 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab\": container with ID starting with a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab not found: ID does not exist" containerID="a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab" Jul 06 18:27:29.215460 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.215421 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab"} err="failed to get container status \"a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab\": rpc error: code = NotFound desc = could not find container \"a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab\": container with ID starting with a5709dbe3f4780214cb0e7fba184e0f94458515df2645a75fa5f4bd4e5b08bab not found: ID does not exist" Jul 06 18:27:29.226115 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.226091 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6fjml"] Jul 06 18:27:29.229507 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.229490 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6fjml"] Jul 06 18:27:29.529658 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:27:29.529625 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45306e8-629d-4915-8598-16c1f3fea49d" path="/var/lib/kubelet/pods/e45306e8-629d-4915-8598-16c1f3fea49d/volumes" Jul 06 18:28:09.221009 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.220978 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-97vr9"] Jul 06 18:28:09.221480 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.221463 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45306e8-629d-4915-8598-16c1f3fea49d" containerName="authorino" Jul 06 18:28:09.221480 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.221477 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45306e8-629d-4915-8598-16c1f3fea49d" containerName="authorino" Jul 06 18:28:09.221573 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.221559 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45306e8-629d-4915-8598-16c1f3fea49d" containerName="authorino" Jul 06 18:28:09.224309 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.224290 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.226839 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.226819 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tt62t\"" Jul 06 18:28:09.226956 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.226847 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 18:28:09.233621 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.233599 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-97vr9"] Jul 06 18:28:09.271924 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.271889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqkp6\" (UniqueName: \"kubernetes.io/projected/9f04b1ab-0dcd-447c-9f31-5aa61200a792-kube-api-access-mqkp6\") pod \"payload-processing-58f86c7798-97vr9\" (UID: \"9f04b1ab-0dcd-447c-9f31-5aa61200a792\") " pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.272093 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.272009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f04b1ab-0dcd-447c-9f31-5aa61200a792-plugins-config-volume\") pod \"payload-processing-58f86c7798-97vr9\" (UID: \"9f04b1ab-0dcd-447c-9f31-5aa61200a792\") " pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.373363 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.373309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f04b1ab-0dcd-447c-9f31-5aa61200a792-plugins-config-volume\") pod \"payload-processing-58f86c7798-97vr9\" (UID: \"9f04b1ab-0dcd-447c-9f31-5aa61200a792\") " pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.373524 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.373369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqkp6\" (UniqueName: \"kubernetes.io/projected/9f04b1ab-0dcd-447c-9f31-5aa61200a792-kube-api-access-mqkp6\") pod \"payload-processing-58f86c7798-97vr9\" (UID: \"9f04b1ab-0dcd-447c-9f31-5aa61200a792\") " pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.373996 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.373976 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f04b1ab-0dcd-447c-9f31-5aa61200a792-plugins-config-volume\") pod \"payload-processing-58f86c7798-97vr9\" (UID: \"9f04b1ab-0dcd-447c-9f31-5aa61200a792\") " pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.383663 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.383628 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqkp6\" (UniqueName: \"kubernetes.io/projected/9f04b1ab-0dcd-447c-9f31-5aa61200a792-kube-api-access-mqkp6\") pod \"payload-processing-58f86c7798-97vr9\" (UID: \"9f04b1ab-0dcd-447c-9f31-5aa61200a792\") " pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.536847 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.536759 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:09.668353 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:09.668311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-97vr9"] Jul 06 18:28:09.671419 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:28:09.671392 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f04b1ab_0dcd_447c_9f31_5aa61200a792.slice/crio-e27d1ae21367803454258d66fa19750475bb9ae25b53a266ef7398d7643b0b9d WatchSource:0}: Error finding container e27d1ae21367803454258d66fa19750475bb9ae25b53a266ef7398d7643b0b9d: Status 404 returned error can't find the container with id e27d1ae21367803454258d66fa19750475bb9ae25b53a266ef7398d7643b0b9d Jul 06 18:28:10.399081 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:10.398999 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" event={"ID":"9f04b1ab-0dcd-447c-9f31-5aa61200a792","Type":"ContainerStarted","Data":"e27d1ae21367803454258d66fa19750475bb9ae25b53a266ef7398d7643b0b9d"} Jul 06 18:28:11.405072 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:11.405034 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" event={"ID":"9f04b1ab-0dcd-447c-9f31-5aa61200a792","Type":"ContainerStarted","Data":"f3f6b8cebb8fbc34d321e0c25dd0ff87102500dc242ce3e4033c05ea42813ad1"} Jul 06 18:28:11.405480 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:11.405175 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:11.421554 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:11.421507 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" podStartSLOduration=0.800057514 podStartE2EDuration="2.421492635s" totalImagesPullingTime="1.621435121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:11.420388503 +0000 UTC m=+622.566929424" watchObservedRunningTime="2026-07-06 18:28:11.421492635 +0000 UTC m=+622.568033565" Jul 06 18:28:22.411674 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:22.411639 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-97vr9" Jul 06 18:28:49.080747 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.080715 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc"] Jul 06 18:28:49.084097 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.084075 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.088369 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.088346 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 18:28:49.088504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.088371 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5psm8\"" Jul 06 18:28:49.088504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.088385 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 18:28:49.088504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.088406 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 18:28:49.097411 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.097348 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc"] Jul 06 18:28:49.147622 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.147592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.147622 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.147628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrvxs\" (UniqueName: \"kubernetes.io/projected/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-kube-api-access-rrvxs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.147864 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.147666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.147864 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.147736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.147864 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.147760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.147864 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.147779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249058 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249058 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrvxs\" (UniqueName: \"kubernetes.io/projected/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-kube-api-access-rrvxs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249276 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249276 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249276 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249276 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249513 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249485 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249571 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249496 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.249679 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.249655 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.251805 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.251782 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.251931 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.251809 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 18:28:49.257851 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.257830 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 18:28:49.261884 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.261862 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.267964 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.267947 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 18:28:49.277939 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.277912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrvxs\" (UniqueName: \"kubernetes.io/projected/e36ad723-cfb9-44f7-b03e-9b6d0269c60c-kube-api-access-rrvxs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5n7mc\" (UID: \"e36ad723-cfb9-44f7-b03e-9b6d0269c60c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.397082 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.396997 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5psm8\"" Jul 06 18:28:49.405256 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.405226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:49.546235 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.546213 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc"] Jul 06 18:28:49.548584 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:28:49.548559 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36ad723_cfb9_44f7_b03e_9b6d0269c60c.slice/crio-e00577af6b3cf793874e8f1b59a9e10ed4c3c477b801907e336b588997fc28f5 WatchSource:0}: Error finding container e00577af6b3cf793874e8f1b59a9e10ed4c3c477b801907e336b588997fc28f5: Status 404 returned error can't find the container with id e00577af6b3cf793874e8f1b59a9e10ed4c3c477b801907e336b588997fc28f5 Jul 06 18:28:49.549872 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.549855 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:28:49.587949 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:49.587917 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" event={"ID":"e36ad723-cfb9-44f7-b03e-9b6d0269c60c","Type":"ContainerStarted","Data":"e00577af6b3cf793874e8f1b59a9e10ed4c3c477b801907e336b588997fc28f5"} Jul 06 18:28:51.599292 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:51.599258 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" event={"ID":"e36ad723-cfb9-44f7-b03e-9b6d0269c60c","Type":"ContainerStarted","Data":"49ca8a754881d4817329197e5c16cade69f29ebfc1f2d1cbc63da31950e79d11"} Jul 06 18:28:51.614816 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:51.614770 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" podStartSLOduration=1.5839118220000001 podStartE2EDuration="2.614754928s" totalImagesPullingTime="1.030843106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:51.614599656 +0000 UTC m=+662.761140579" watchObservedRunningTime="2026-07-06 18:28:51.614754928 +0000 UTC m=+662.761296207" Jul 06 18:28:59.406127 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:59.406082 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:59.406615 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:59.406152 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:59.419287 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:59.419262 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:28:59.647967 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:28:59.647937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5n7mc" Jul 06 18:29:00.592733 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.592697 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg"] Jul 06 18:29:00.596432 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.596412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.598977 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.598960 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 18:29:00.606111 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.606064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg"] Jul 06 18:29:00.657756 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.657709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2366-d3e7-438e-ba40-f6259b95f538-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.657756 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.657754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gghkc\" (UniqueName: \"kubernetes.io/projected/8f6d2366-d3e7-438e-ba40-f6259b95f538-kube-api-access-gghkc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.658006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.657803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.658006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.657862 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.658006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.657917 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.658006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.657945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759089 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759048 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2366-d3e7-438e-ba40-f6259b95f538-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759089 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gghkc\" (UniqueName: \"kubernetes.io/projected/8f6d2366-d3e7-438e-ba40-f6259b95f538-kube-api-access-gghkc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759388 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759119 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759388 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759171 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759388 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759388 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759605 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759585 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759714 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759691 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.759754 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.759725 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.761510 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.761489 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f6d2366-d3e7-438e-ba40-f6259b95f538-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.761750 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.761730 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2366-d3e7-438e-ba40-f6259b95f538-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.767433 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.767409 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gghkc\" (UniqueName: \"kubernetes.io/projected/8f6d2366-d3e7-438e-ba40-f6259b95f538-kube-api-access-gghkc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg\" (UID: \"8f6d2366-d3e7-438e-ba40-f6259b95f538\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:00.907541 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:00.907451 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:01.038726 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:01.038703 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg"] Jul 06 18:29:01.041432 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:29:01.041399 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6d2366_d3e7_438e_ba40_f6259b95f538.slice/crio-86354fa3c1d23f06db2fbee5e6ffcfdb75aae83fa0bab98a8c2b17e64081fde6 WatchSource:0}: Error finding container 86354fa3c1d23f06db2fbee5e6ffcfdb75aae83fa0bab98a8c2b17e64081fde6: Status 404 returned error can't find the container with id 86354fa3c1d23f06db2fbee5e6ffcfdb75aae83fa0bab98a8c2b17e64081fde6 Jul 06 18:29:01.649038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:01.648934 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" event={"ID":"8f6d2366-d3e7-438e-ba40-f6259b95f538","Type":"ContainerStarted","Data":"e8a4ecd27e0bfe540a1d1bae283212b08db4eab4f1ca56d0113b40a6795e40af"} Jul 06 18:29:01.649038 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:01.648996 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" event={"ID":"8f6d2366-d3e7-438e-ba40-f6259b95f538","Type":"ContainerStarted","Data":"86354fa3c1d23f06db2fbee5e6ffcfdb75aae83fa0bab98a8c2b17e64081fde6"} Jul 06 18:29:01.672731 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:01.671810 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" podStartSLOduration=1.49975363 podStartE2EDuration="1.671789408s" totalImagesPullingTime="172.035778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:29:01.668624399 +0000 UTC m=+672.815165322" watchObservedRunningTime="2026-07-06 18:29:01.671789408 +0000 UTC m=+672.818330334" Jul 06 18:29:10.908037 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:10.907993 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:10.908564 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:10.908053 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:10.920908 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:10.920882 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:11.709138 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:11.709107 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg" Jul 06 18:29:25.288401 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.288367 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8"] Jul 06 18:29:25.293795 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.293766 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.296299 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.296278 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 18:29:25.302219 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.302198 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8"] Jul 06 18:29:25.392027 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.391989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1156af4a-b8ae-4c3d-b845-147530a24b49-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.392027 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.392033 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.392232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.392052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.392232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.392125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.392232 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.392147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.392354 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.392299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrbhg\" (UniqueName: \"kubernetes.io/projected/1156af4a-b8ae-4c3d-b845-147530a24b49-kube-api-access-xrbhg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.493771 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.493717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrbhg\" (UniqueName: \"kubernetes.io/projected/1156af4a-b8ae-4c3d-b845-147530a24b49-kube-api-access-xrbhg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.493958 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.493785 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1156af4a-b8ae-4c3d-b845-147530a24b49-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.493958 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.493819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.493958 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.493844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.493958 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.493877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.494144 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.494004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.494288 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.494267 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.494402 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.494310 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.494463 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.494431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.496208 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.496191 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1156af4a-b8ae-4c3d-b845-147530a24b49-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.496513 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.496493 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1156af4a-b8ae-4c3d-b845-147530a24b49-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.501499 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.501473 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrbhg\" (UniqueName: \"kubernetes.io/projected/1156af4a-b8ae-4c3d-b845-147530a24b49-kube-api-access-xrbhg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8\" (UID: \"1156af4a-b8ae-4c3d-b845-147530a24b49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.604414 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.604313 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:25.740078 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.739991 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8"] Jul 06 18:29:25.742498 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:29:25.742472 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1156af4a_b8ae_4c3d_b845_147530a24b49.slice/crio-ed511676e344baeaae861bb1705a3cf22642ff1723b8b0375c1e70ad28e54836 WatchSource:0}: Error finding container ed511676e344baeaae861bb1705a3cf22642ff1723b8b0375c1e70ad28e54836: Status 404 returned error can't find the container with id ed511676e344baeaae861bb1705a3cf22642ff1723b8b0375c1e70ad28e54836 Jul 06 18:29:25.768465 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:25.768427 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" event={"ID":"1156af4a-b8ae-4c3d-b845-147530a24b49","Type":"ContainerStarted","Data":"ed511676e344baeaae861bb1705a3cf22642ff1723b8b0375c1e70ad28e54836"} Jul 06 18:29:26.775340 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:26.775279 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" event={"ID":"1156af4a-b8ae-4c3d-b845-147530a24b49","Type":"ContainerStarted","Data":"116fa5415f49c3f4bd98f65f4e315b56a15f6fabf121db322bf7201b6ab1a66d"} Jul 06 18:29:26.794353 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:26.794291 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" podStartSLOduration=1.600111805 podStartE2EDuration="1.794275493s" totalImagesPullingTime="194.163688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:29:26.792731465 +0000 UTC m=+697.939272388" watchObservedRunningTime="2026-07-06 18:29:26.794275493 +0000 UTC m=+697.940816414" Jul 06 18:29:35.605203 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:35.605167 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:35.605646 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:35.605224 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:35.618305 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:35.618280 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:29:35.831978 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:29:35.831950 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8" Jul 06 18:36:21.348932 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:21.348898 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649"] Jul 06 18:36:21.349535 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:21.349090 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" podUID="85de066e-1490-4c28-ae3b-11fd32d7e5f4" containerName="manager" containerID="cri-o://055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676" gracePeriod=10 Jul 06 18:36:23.398387 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.398361 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:36:23.434774 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.434739 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85de066e-1490-4c28-ae3b-11fd32d7e5f4-extensions-socket-volume\") pod \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " Jul 06 18:36:23.434964 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.434796 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzg49\" (UniqueName: \"kubernetes.io/projected/85de066e-1490-4c28-ae3b-11fd32d7e5f4-kube-api-access-pzg49\") pod \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\" (UID: \"85de066e-1490-4c28-ae3b-11fd32d7e5f4\") " Jul 06 18:36:23.435155 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.435131 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85de066e-1490-4c28-ae3b-11fd32d7e5f4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "85de066e-1490-4c28-ae3b-11fd32d7e5f4" (UID: "85de066e-1490-4c28-ae3b-11fd32d7e5f4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:36:23.436917 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.436882 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85de066e-1490-4c28-ae3b-11fd32d7e5f4-kube-api-access-pzg49" (OuterVolumeSpecName: "kube-api-access-pzg49") pod "85de066e-1490-4c28-ae3b-11fd32d7e5f4" (UID: "85de066e-1490-4c28-ae3b-11fd32d7e5f4"). InnerVolumeSpecName "kube-api-access-pzg49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:36:23.536254 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.535997 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85de066e-1490-4c28-ae3b-11fd32d7e5f4-extensions-socket-volume\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:36:23.536254 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.536024 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzg49\" (UniqueName: \"kubernetes.io/projected/85de066e-1490-4c28-ae3b-11fd32d7e5f4-kube-api-access-pzg49\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:36:23.818070 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.817982 2566 generic.go:356] "Generic (PLEG): container finished" podID="85de066e-1490-4c28-ae3b-11fd32d7e5f4" containerID="055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676" exitCode=0 Jul 06 18:36:23.818070 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.818032 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" Jul 06 18:36:23.818070 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.818040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" event={"ID":"85de066e-1490-4c28-ae3b-11fd32d7e5f4","Type":"ContainerDied","Data":"055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676"} Jul 06 18:36:23.818284 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.818091 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649" event={"ID":"85de066e-1490-4c28-ae3b-11fd32d7e5f4","Type":"ContainerDied","Data":"67578b5225fcb6636ca62d517ab358c7803b6b8ce5f5fdc095493ce110609606"} Jul 06 18:36:23.818284 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.818121 2566 scope.go:117] "RemoveContainer" containerID="055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676" Jul 06 18:36:23.828128 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.828108 2566 scope.go:117] "RemoveContainer" containerID="055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676" Jul 06 18:36:23.828392 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:36:23.828371 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676\": container with ID starting with 055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676 not found: ID does not exist" containerID="055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676" Jul 06 18:36:23.828437 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.828407 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676"} err="failed to get container status \"055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676\": rpc error: code = NotFound desc = could not find container \"055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676\": container with ID starting with 055417cf12168f72b03d3bd677860509b7b06ccbc589ca5315f3a6dbfafee676 not found: ID does not exist" Jul 06 18:36:23.835695 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.835670 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649"] Jul 06 18:36:23.839215 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:23.839194 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k649"] Jul 06 18:36:25.529212 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:36:25.529181 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85de066e-1490-4c28-ae3b-11fd32d7e5f4" path="/var/lib/kubelet/pods/85de066e-1490-4c28-ae3b-11fd32d7e5f4/volumes" Jul 06 18:37:22.549884 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.549846 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt"] Jul 06 18:37:22.550396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.550244 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85de066e-1490-4c28-ae3b-11fd32d7e5f4" containerName="manager" Jul 06 18:37:22.550396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.550256 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="85de066e-1490-4c28-ae3b-11fd32d7e5f4" containerName="manager" Jul 06 18:37:22.550396 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.550353 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="85de066e-1490-4c28-ae3b-11fd32d7e5f4" containerName="manager" Jul 06 18:37:22.553299 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.553281 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.558233 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.558213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v4csq\"" Jul 06 18:37:22.576308 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.576284 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt"] Jul 06 18:37:22.694523 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.694490 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwg4c\" (UniqueName: \"kubernetes.io/projected/6ad0f006-50d1-4a0b-8815-d78ba411c169-kube-api-access-zwg4c\") pod \"kuadrant-operator-controller-manager-86878c769f-8bszt\" (UID: \"6ad0f006-50d1-4a0b-8815-d78ba411c169\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.694701 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.694547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ad0f006-50d1-4a0b-8815-d78ba411c169-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8bszt\" (UID: \"6ad0f006-50d1-4a0b-8815-d78ba411c169\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.795647 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.795612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ad0f006-50d1-4a0b-8815-d78ba411c169-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8bszt\" (UID: \"6ad0f006-50d1-4a0b-8815-d78ba411c169\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.795842 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.795729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwg4c\" (UniqueName: \"kubernetes.io/projected/6ad0f006-50d1-4a0b-8815-d78ba411c169-kube-api-access-zwg4c\") pod \"kuadrant-operator-controller-manager-86878c769f-8bszt\" (UID: \"6ad0f006-50d1-4a0b-8815-d78ba411c169\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.796006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.795985 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ad0f006-50d1-4a0b-8815-d78ba411c169-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8bszt\" (UID: \"6ad0f006-50d1-4a0b-8815-d78ba411c169\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.807301 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.807226 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwg4c\" (UniqueName: \"kubernetes.io/projected/6ad0f006-50d1-4a0b-8815-d78ba411c169-kube-api-access-zwg4c\") pod \"kuadrant-operator-controller-manager-86878c769f-8bszt\" (UID: \"6ad0f006-50d1-4a0b-8815-d78ba411c169\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:22.864305 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:22.864273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:23.022279 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:23.020884 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt"] Jul 06 18:37:23.025072 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:37:23.025046 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad0f006_50d1_4a0b_8815_d78ba411c169.slice/crio-e11893ce79691920d1d31c48180b8302a6e91e9301fc5ec29237d0eca581abcf WatchSource:0}: Error finding container e11893ce79691920d1d31c48180b8302a6e91e9301fc5ec29237d0eca581abcf: Status 404 returned error can't find the container with id e11893ce79691920d1d31c48180b8302a6e91e9301fc5ec29237d0eca581abcf Jul 06 18:37:23.101765 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:23.101735 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" event={"ID":"6ad0f006-50d1-4a0b-8815-d78ba411c169","Type":"ContainerStarted","Data":"b153b5ff70c5f8e6004178773072e5f4196af63fce5bb9d6cdd2729902d5dbd7"} Jul 06 18:37:23.101901 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:23.101798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" event={"ID":"6ad0f006-50d1-4a0b-8815-d78ba411c169","Type":"ContainerStarted","Data":"e11893ce79691920d1d31c48180b8302a6e91e9301fc5ec29237d0eca581abcf"} Jul 06 18:37:23.101901 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:23.101863 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:37:23.140302 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:23.140250 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" podStartSLOduration=1.140235792 podStartE2EDuration="1.140235792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:37:23.138709758 +0000 UTC m=+1174.285250680" watchObservedRunningTime="2026-07-06 18:37:23.140235792 +0000 UTC m=+1174.286776713" Jul 06 18:37:34.109114 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:37:34.109084 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bszt" Jul 06 18:50:10.603880 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.603852 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f"] Jul 06 18:50:10.607631 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.607614 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.611484 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.611453 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-aea86e\"/\"kube-root-ca.crt\"" Jul 06 18:50:10.611624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.611488 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-aea86e\"/\"default-dockercfg-dhw4q\"" Jul 06 18:50:10.611624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.611495 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-aea86e\"/\"openshift-service-ca.crt\"" Jul 06 18:50:10.611624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.611589 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-aea86e\"/\"test-model-aea86e-kserve-self-signed-certs\"" Jul 06 18:50:10.617461 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.617440 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f"] Jul 06 18:50:10.725932 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.725894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-dshm\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.725932 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.725937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-tmp-dir\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.726176 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.725977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/052d40fa-e844-438c-b3af-8b50ac33d21d-tls-certs\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.726176 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.725998 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qb2w\" (UniqueName: \"kubernetes.io/projected/052d40fa-e844-438c-b3af-8b50ac33d21d-kube-api-access-5qb2w\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.726176 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.726017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-model-cache\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.726176 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.726032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-home\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.826692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.826655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-dshm\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.826692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.826699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-tmp-dir\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.826899 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.826739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/052d40fa-e844-438c-b3af-8b50ac33d21d-tls-certs\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.826899 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.826773 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qb2w\" (UniqueName: \"kubernetes.io/projected/052d40fa-e844-438c-b3af-8b50ac33d21d-kube-api-access-5qb2w\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.826899 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.826799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-model-cache\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.826899 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.826823 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-home\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.827189 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.827164 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-tmp-dir\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.827246 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.827216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-home\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.827246 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.827234 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-model-cache\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.829137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.829105 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-dshm\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.829262 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.829243 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/052d40fa-e844-438c-b3af-8b50ac33d21d-tls-certs\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.836567 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.836541 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qb2w\" (UniqueName: \"kubernetes.io/projected/052d40fa-e844-438c-b3af-8b50ac33d21d-kube-api-access-5qb2w\") pod \"test-model-aea86e-kserve-8b4498dcc-2sf9f\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:10.919302 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:10.919199 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:11.056100 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:11.056012 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f"] Jul 06 18:50:11.058604 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:50:11.058562 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod052d40fa_e844_438c_b3af_8b50ac33d21d.slice/crio-2959653b2e446b676188fbbf70d6567450450a8cb5adae4dcc4ca751cc87361e WatchSource:0}: Error finding container 2959653b2e446b676188fbbf70d6567450450a8cb5adae4dcc4ca751cc87361e: Status 404 returned error can't find the container with id 2959653b2e446b676188fbbf70d6567450450a8cb5adae4dcc4ca751cc87361e Jul 06 18:50:11.878012 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:11.877974 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" event={"ID":"052d40fa-e844-438c-b3af-8b50ac33d21d","Type":"ContainerStarted","Data":"4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572"} Jul 06 18:50:11.878012 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:11.878017 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" event={"ID":"052d40fa-e844-438c-b3af-8b50ac33d21d","Type":"ContainerStarted","Data":"2959653b2e446b676188fbbf70d6567450450a8cb5adae4dcc4ca751cc87361e"} Jul 06 18:50:11.899220 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:11.899172 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" podStartSLOduration=1.8991563409999999 podStartE2EDuration="1.899156341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:50:11.8972106 +0000 UTC m=+1943.043751523" watchObservedRunningTime="2026-07-06 18:50:11.899156341 +0000 UTC m=+1943.045697266" Jul 06 18:50:20.919748 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:20.919714 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:20.920361 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:20.919804 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:20.934622 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:20.934591 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:21.942484 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:21.942448 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:32.392357 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:32.392307 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f"] Jul 06 18:50:47.413165 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.413115 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" podUID="052d40fa-e844-438c-b3af-8b50ac33d21d" containerName="main" containerID="cri-o://4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572" gracePeriod=45 Jul 06 18:50:47.679230 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.679203 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:47.800946 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.800907 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-tmp-dir\") pod \"052d40fa-e844-438c-b3af-8b50ac33d21d\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " Jul 06 18:50:47.800946 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.800951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-home\") pod \"052d40fa-e844-438c-b3af-8b50ac33d21d\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " Jul 06 18:50:47.801241 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801048 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qb2w\" (UniqueName: \"kubernetes.io/projected/052d40fa-e844-438c-b3af-8b50ac33d21d-kube-api-access-5qb2w\") pod \"052d40fa-e844-438c-b3af-8b50ac33d21d\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " Jul 06 18:50:47.801241 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801076 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/052d40fa-e844-438c-b3af-8b50ac33d21d-tls-certs\") pod \"052d40fa-e844-438c-b3af-8b50ac33d21d\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " Jul 06 18:50:47.801241 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801122 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-dshm\") pod \"052d40fa-e844-438c-b3af-8b50ac33d21d\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " Jul 06 18:50:47.801241 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801170 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-model-cache\") pod \"052d40fa-e844-438c-b3af-8b50ac33d21d\" (UID: \"052d40fa-e844-438c-b3af-8b50ac33d21d\") " Jul 06 18:50:47.801495 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801253 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-home" (OuterVolumeSpecName: "home") pod "052d40fa-e844-438c-b3af-8b50ac33d21d" (UID: "052d40fa-e844-438c-b3af-8b50ac33d21d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:47.801495 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801270 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "052d40fa-e844-438c-b3af-8b50ac33d21d" (UID: "052d40fa-e844-438c-b3af-8b50ac33d21d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:47.801495 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801472 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-model-cache" (OuterVolumeSpecName: "model-cache") pod "052d40fa-e844-438c-b3af-8b50ac33d21d" (UID: "052d40fa-e844-438c-b3af-8b50ac33d21d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:47.801628 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801511 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-tmp-dir\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:50:47.801628 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.801524 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-home\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:50:47.803471 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.803438 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-dshm" (OuterVolumeSpecName: "dshm") pod "052d40fa-e844-438c-b3af-8b50ac33d21d" (UID: "052d40fa-e844-438c-b3af-8b50ac33d21d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:47.803471 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.803441 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/052d40fa-e844-438c-b3af-8b50ac33d21d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "052d40fa-e844-438c-b3af-8b50ac33d21d" (UID: "052d40fa-e844-438c-b3af-8b50ac33d21d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:50:47.803614 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.803511 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052d40fa-e844-438c-b3af-8b50ac33d21d-kube-api-access-5qb2w" (OuterVolumeSpecName: "kube-api-access-5qb2w") pod "052d40fa-e844-438c-b3af-8b50ac33d21d" (UID: "052d40fa-e844-438c-b3af-8b50ac33d21d"). InnerVolumeSpecName "kube-api-access-5qb2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:50:47.902836 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.902798 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qb2w\" (UniqueName: \"kubernetes.io/projected/052d40fa-e844-438c-b3af-8b50ac33d21d-kube-api-access-5qb2w\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:50:47.902836 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.902828 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/052d40fa-e844-438c-b3af-8b50ac33d21d-tls-certs\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:50:47.902836 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.902837 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-dshm\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:50:47.902836 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:47.902846 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/052d40fa-e844-438c-b3af-8b50ac33d21d-model-cache\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:50:48.065251 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.065217 2566 generic.go:356] "Generic (PLEG): container finished" podID="052d40fa-e844-438c-b3af-8b50ac33d21d" containerID="4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572" exitCode=0 Jul 06 18:50:48.065443 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.065266 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" Jul 06 18:50:48.065443 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.065274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" event={"ID":"052d40fa-e844-438c-b3af-8b50ac33d21d","Type":"ContainerDied","Data":"4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572"} Jul 06 18:50:48.065443 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.065307 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f" event={"ID":"052d40fa-e844-438c-b3af-8b50ac33d21d","Type":"ContainerDied","Data":"2959653b2e446b676188fbbf70d6567450450a8cb5adae4dcc4ca751cc87361e"} Jul 06 18:50:48.065443 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.065354 2566 scope.go:117] "RemoveContainer" containerID="4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572" Jul 06 18:50:48.074678 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.074661 2566 scope.go:117] "RemoveContainer" containerID="4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572" Jul 06 18:50:48.074929 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:50:48.074910 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572\": container with ID starting with 4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572 not found: ID does not exist" containerID="4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572" Jul 06 18:50:48.074977 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.074936 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572"} err="failed to get container status \"4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572\": rpc error: code = NotFound desc = could not find container \"4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572\": container with ID starting with 4d0b8a3b2788bd49155839dbb7b006e6615a779a99015be130366c9392a9e572 not found: ID does not exist" Jul 06 18:50:48.085459 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.085436 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f"] Jul 06 18:50:48.089430 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:48.089409 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-aea86e/test-model-aea86e-kserve-8b4498dcc-2sf9f"] Jul 06 18:50:49.529837 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:50:49.529800 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="052d40fa-e844-438c-b3af-8b50ac33d21d" path="/var/lib/kubelet/pods/052d40fa-e844-438c-b3af-8b50ac33d21d/volumes" Jul 06 18:52:55.289415 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.289382 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw"] Jul 06 18:52:55.289965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.289824 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="052d40fa-e844-438c-b3af-8b50ac33d21d" containerName="main" Jul 06 18:52:55.289965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.289843 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="052d40fa-e844-438c-b3af-8b50ac33d21d" containerName="main" Jul 06 18:52:55.290069 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.289991 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="052d40fa-e844-438c-b3af-8b50ac33d21d" containerName="main" Jul 06 18:52:55.293150 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.293135 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.296606 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.296585 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"kube-root-ca.crt\"" Jul 06 18:52:55.296606 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.296593 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"default-dockercfg-7gff7\"" Jul 06 18:52:55.296773 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.296638 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"openshift-service-ca.crt\"" Jul 06 18:52:55.296773 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.296593 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"auth-test-model-b67177-kserve-self-signed-certs\"" Jul 06 18:52:55.300245 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.300221 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw"] Jul 06 18:52:55.434753 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.434720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-model-cache\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.434943 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.434781 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-dshm\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.434943 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.434883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tmp-dir\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.434943 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.434917 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-home\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.434943 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.434938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzv4k\" (UniqueName: \"kubernetes.io/projected/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-kube-api-access-nzv4k\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.435073 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.434957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tls-certs\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.535853 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.535818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-model-cache\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536030 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.535900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-dshm\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536030 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.535956 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tmp-dir\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536030 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.535981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-home\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536030 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.536009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzv4k\" (UniqueName: \"kubernetes.io/projected/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-kube-api-access-nzv4k\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536235 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.536039 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tls-certs\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536294 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.536272 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-model-cache\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536294 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.536286 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tmp-dir\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.536445 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.536423 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-home\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.538180 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.538151 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-dshm\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.538657 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.538637 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tls-certs\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.544492 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.544429 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzv4k\" (UniqueName: \"kubernetes.io/projected/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-kube-api-access-nzv4k\") pod \"auth-test-model-b67177-kserve-7bb685bd8c-mzfxw\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.603564 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.603528 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:52:55.744012 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:55.743988 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw"] Jul 06 18:52:55.746960 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:52:55.746934 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59202f4b_66d0_4fbc_b16a_1e9479d2ae3f.slice/crio-3310a33261b8fbac47fe04b82e9d5c5385fee35959d591ae382559650b68db50 WatchSource:0}: Error finding container 3310a33261b8fbac47fe04b82e9d5c5385fee35959d591ae382559650b68db50: Status 404 returned error can't find the container with id 3310a33261b8fbac47fe04b82e9d5c5385fee35959d591ae382559650b68db50 Jul 06 18:52:56.690068 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:56.690033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" event={"ID":"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f","Type":"ContainerStarted","Data":"58e415125540b70535146eaa0c0d404e55efd649105ce5aa3b3d87e1b0c95ec4"} Jul 06 18:52:56.690445 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:56.690077 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" event={"ID":"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f","Type":"ContainerStarted","Data":"3310a33261b8fbac47fe04b82e9d5c5385fee35959d591ae382559650b68db50"} Jul 06 18:52:56.708745 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:52:56.708699 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" podStartSLOduration=1.70868369 podStartE2EDuration="1.70868369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:52:56.70696781 +0000 UTC m=+2107.853508732" watchObservedRunningTime="2026-07-06 18:52:56.70868369 +0000 UTC m=+2107.855224612" Jul 06 18:53:05.603773 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:53:05.603737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:53:05.604178 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:53:05.603812 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:53:05.616348 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:53:05.616298 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:53:05.745438 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:53:05.745406 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:54:50.097372 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:54:50.097304 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw"] Jul 06 18:55:05.117834 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.117784 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" podUID="59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" containerName="main" containerID="cri-o://58e415125540b70535146eaa0c0d404e55efd649105ce5aa3b3d87e1b0c95ec4" gracePeriod=45 Jul 06 18:55:05.325712 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.325681 2566 generic.go:356] "Generic (PLEG): container finished" podID="59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" containerID="58e415125540b70535146eaa0c0d404e55efd649105ce5aa3b3d87e1b0c95ec4" exitCode=0 Jul 06 18:55:05.325882 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.325722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" event={"ID":"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f","Type":"ContainerDied","Data":"58e415125540b70535146eaa0c0d404e55efd649105ce5aa3b3d87e1b0c95ec4"} Jul 06 18:55:05.371593 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.371532 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:55:05.506845 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.506811 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-home\") pod \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " Jul 06 18:55:05.507009 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.506859 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tmp-dir\") pod \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " Jul 06 18:55:05.507009 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.506923 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tls-certs\") pod \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " Jul 06 18:55:05.507009 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.506961 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzv4k\" (UniqueName: \"kubernetes.io/projected/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-kube-api-access-nzv4k\") pod \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " Jul 06 18:55:05.507009 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.506995 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-model-cache\") pod \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " Jul 06 18:55:05.507237 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507070 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-dshm\") pod \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\" (UID: \"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f\") " Jul 06 18:55:05.507237 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507066 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-home" (OuterVolumeSpecName: "home") pod "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" (UID: "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:55:05.507237 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507176 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" (UID: "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:55:05.507428 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507244 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" (UID: "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:55:05.507487 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507430 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-home\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:55:05.507487 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507448 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tmp-dir\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:55:05.507487 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.507458 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-model-cache\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:55:05.509185 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.509162 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-dshm" (OuterVolumeSpecName: "dshm") pod "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" (UID: "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:55:05.509271 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.509209 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" (UID: "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:55:05.509379 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.509294 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-kube-api-access-nzv4k" (OuterVolumeSpecName: "kube-api-access-nzv4k") pod "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" (UID: "59202f4b-66d0-4fbc-b16a-1e9479d2ae3f"). InnerVolumeSpecName "kube-api-access-nzv4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:55:05.608826 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.608797 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzv4k\" (UniqueName: \"kubernetes.io/projected/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-kube-api-access-nzv4k\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:55:05.608826 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.608822 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-dshm\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:55:05.608826 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:05.608832 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f-tls-certs\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:55:06.335269 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:06.335232 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" Jul 06 18:55:06.335745 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:06.335266 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw" event={"ID":"59202f4b-66d0-4fbc-b16a-1e9479d2ae3f","Type":"ContainerDied","Data":"3310a33261b8fbac47fe04b82e9d5c5385fee35959d591ae382559650b68db50"} Jul 06 18:55:06.335745 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:06.335343 2566 scope.go:117] "RemoveContainer" containerID="58e415125540b70535146eaa0c0d404e55efd649105ce5aa3b3d87e1b0c95ec4" Jul 06 18:55:06.352361 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:06.352335 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw"] Jul 06 18:55:06.356643 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:06.356610 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/auth-test-model-b67177-kserve-7bb685bd8c-mzfxw"] Jul 06 18:55:07.530016 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:07.529976 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" path="/var/lib/kubelet/pods/59202f4b-66d0-4fbc-b16a-1e9479d2ae3f/volumes" Jul 06 18:55:14.084866 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.084791 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp"] Jul 06 18:55:14.085240 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.085178 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" containerName="main" Jul 06 18:55:14.085240 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.085190 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" containerName="main" Jul 06 18:55:14.085347 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.085280 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="59202f4b-66d0-4fbc-b16a-1e9479d2ae3f" containerName="main" Jul 06 18:55:14.089509 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.089486 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.092167 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.092143 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"kube-root-ca.crt\"" Jul 06 18:55:14.093129 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.093116 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"openshift-service-ca.crt\"" Jul 06 18:55:14.093368 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.093355 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"default-dockercfg-7gff7\"" Jul 06 18:55:14.093453 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.093438 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"sub-test-model-b67177-kserve-self-signed-certs\"" Jul 06 18:55:14.097137 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.097114 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp"] Jul 06 18:55:14.192161 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.192126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tmp-dir\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.192382 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.192167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knv5w\" (UniqueName: \"kubernetes.io/projected/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-kube-api-access-knv5w\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.192382 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.192188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tls-certs\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.192382 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.192303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-dshm\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.192526 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.192379 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-home\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.192526 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.192406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-model-cache\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293237 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tmp-dir\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293453 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293246 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knv5w\" (UniqueName: \"kubernetes.io/projected/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-kube-api-access-knv5w\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293453 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293266 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tls-certs\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293453 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-dshm\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293453 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-home\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293453 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-model-cache\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293774 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293755 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tmp-dir\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293841 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-home\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.293841 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.293832 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-model-cache\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.295707 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.295685 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-dshm\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.295800 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.295781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tls-certs\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.303620 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.303592 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knv5w\" (UniqueName: \"kubernetes.io/projected/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-kube-api-access-knv5w\") pod \"sub-test-model-b67177-kserve-6d49b5967f-2w7qp\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.400016 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.399922 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:14.535081 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:14.535057 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp"] Jul 06 18:55:14.537298 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:55:14.537269 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod396ee576_3af2_4ddf_8b33_b4cc669f7b1b.slice/crio-20a2896617be6ef84e547c91f4c4b68c4ff27e0acc336f9673205f6fce74ab90 WatchSource:0}: Error finding container 20a2896617be6ef84e547c91f4c4b68c4ff27e0acc336f9673205f6fce74ab90: Status 404 returned error can't find the container with id 20a2896617be6ef84e547c91f4c4b68c4ff27e0acc336f9673205f6fce74ab90 Jul 06 18:55:15.382557 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:15.382516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" event={"ID":"396ee576-3af2-4ddf-8b33-b4cc669f7b1b","Type":"ContainerStarted","Data":"c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b"} Jul 06 18:55:15.382557 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:15.382562 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" event={"ID":"396ee576-3af2-4ddf-8b33-b4cc669f7b1b","Type":"ContainerStarted","Data":"20a2896617be6ef84e547c91f4c4b68c4ff27e0acc336f9673205f6fce74ab90"} Jul 06 18:55:15.401465 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:15.401417 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" podStartSLOduration=1.401402051 podStartE2EDuration="1.401402051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:15.40076602 +0000 UTC m=+2246.547306954" watchObservedRunningTime="2026-07-06 18:55:15.401402051 +0000 UTC m=+2246.547942973" Jul 06 18:55:24.401103 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:24.401066 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:24.401103 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:24.401111 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:24.414623 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:24.414598 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:24.441493 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:24.441459 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:55:52.510007 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:55:52.509973 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp"] Jul 06 18:56:03.127848 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.127805 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872"] Jul 06 18:56:03.132491 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.132471 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.137437 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.137416 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"openshift-service-ca.crt\"" Jul 06 18:56:03.137664 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.137447 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"kube-root-ca.crt\"" Jul 06 18:56:03.137664 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.137450 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"rate-test-model-8c615a-kserve-self-signed-certs\"" Jul 06 18:56:03.137848 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.137484 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"default-dockercfg-s7fj4\"" Jul 06 18:56:03.142369 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.142306 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872"] Jul 06 18:56:03.177252 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.177217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-tmp-dir\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.177439 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.177260 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ce691d-2172-4069-8ee0-8f1bbede5a89-tls-certs\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.177439 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.177292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-home\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.177439 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.177381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-dshm\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.177439 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.177400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j57q5\" (UniqueName: \"kubernetes.io/projected/79ce691d-2172-4069-8ee0-8f1bbede5a89-kube-api-access-j57q5\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.177439 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.177439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-model-cache\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278444 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-dshm\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278444 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278448 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j57q5\" (UniqueName: \"kubernetes.io/projected/79ce691d-2172-4069-8ee0-8f1bbede5a89-kube-api-access-j57q5\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278667 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-model-cache\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278667 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-tmp-dir\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278667 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ce691d-2172-4069-8ee0-8f1bbede5a89-tls-certs\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278667 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-home\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.278929 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278905 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-model-cache\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.279018 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.278931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-home\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.279070 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.279017 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-tmp-dir\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.280780 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.280760 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-dshm\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.280959 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.280942 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ce691d-2172-4069-8ee0-8f1bbede5a89-tls-certs\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.287351 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.287289 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j57q5\" (UniqueName: \"kubernetes.io/projected/79ce691d-2172-4069-8ee0-8f1bbede5a89-kube-api-access-j57q5\") pod \"rate-test-model-8c615a-kserve-f7c5d747-tl872\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.443536 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.443444 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:03.588884 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.588861 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872"] Jul 06 18:56:03.590754 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:56:03.590730 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79ce691d_2172_4069_8ee0_8f1bbede5a89.slice/crio-975a2882f6949a55a58cb728e996f3fbbbc7d281662593e2056c50eb871c418a WatchSource:0}: Error finding container 975a2882f6949a55a58cb728e996f3fbbbc7d281662593e2056c50eb871c418a: Status 404 returned error can't find the container with id 975a2882f6949a55a58cb728e996f3fbbbc7d281662593e2056c50eb871c418a Jul 06 18:56:03.620504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:03.620473 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" event={"ID":"79ce691d-2172-4069-8ee0-8f1bbede5a89","Type":"ContainerStarted","Data":"975a2882f6949a55a58cb728e996f3fbbbc7d281662593e2056c50eb871c418a"} Jul 06 18:56:04.626753 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:04.626719 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" event={"ID":"79ce691d-2172-4069-8ee0-8f1bbede5a89","Type":"ContainerStarted","Data":"f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02"} Jul 06 18:56:04.644706 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:04.644653 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" podStartSLOduration=1.64463743 podStartE2EDuration="1.64463743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:04.644180371 +0000 UTC m=+2295.790721292" watchObservedRunningTime="2026-07-06 18:56:04.64463743 +0000 UTC m=+2295.791178352" Jul 06 18:56:07.531016 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.530972 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" podUID="396ee576-3af2-4ddf-8b33-b4cc669f7b1b" containerName="main" containerID="cri-o://c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b" gracePeriod=45 Jul 06 18:56:07.782635 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.782577 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:56:07.922950 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.922916 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-model-cache\") pod \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " Jul 06 18:56:07.923122 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.922985 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-dshm\") pod \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " Jul 06 18:56:07.923122 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923035 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tmp-dir\") pod \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " Jul 06 18:56:07.923122 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923060 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-home\") pod \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " Jul 06 18:56:07.923250 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923125 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knv5w\" (UniqueName: \"kubernetes.io/projected/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-kube-api-access-knv5w\") pod \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " Jul 06 18:56:07.923250 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923145 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tls-certs\") pod \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\" (UID: \"396ee576-3af2-4ddf-8b33-b4cc669f7b1b\") " Jul 06 18:56:07.923389 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923248 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "396ee576-3af2-4ddf-8b33-b4cc669f7b1b" (UID: "396ee576-3af2-4ddf-8b33-b4cc669f7b1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:56:07.923389 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923279 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "396ee576-3af2-4ddf-8b33-b4cc669f7b1b" (UID: "396ee576-3af2-4ddf-8b33-b4cc669f7b1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:56:07.923504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923423 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-home" (OuterVolumeSpecName: "home") pod "396ee576-3af2-4ddf-8b33-b4cc669f7b1b" (UID: "396ee576-3af2-4ddf-8b33-b4cc669f7b1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:56:07.923558 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923515 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-model-cache\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:56:07.923558 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923535 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tmp-dir\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:56:07.923558 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.923548 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-home\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:56:07.925255 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.925227 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "396ee576-3af2-4ddf-8b33-b4cc669f7b1b" (UID: "396ee576-3af2-4ddf-8b33-b4cc669f7b1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:56:07.925388 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.925264 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-dshm" (OuterVolumeSpecName: "dshm") pod "396ee576-3af2-4ddf-8b33-b4cc669f7b1b" (UID: "396ee576-3af2-4ddf-8b33-b4cc669f7b1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:56:07.925680 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:07.925447 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-kube-api-access-knv5w" (OuterVolumeSpecName: "kube-api-access-knv5w") pod "396ee576-3af2-4ddf-8b33-b4cc669f7b1b" (UID: "396ee576-3af2-4ddf-8b33-b4cc669f7b1b"). InnerVolumeSpecName "kube-api-access-knv5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:56:08.024461 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.024421 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-dshm\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:56:08.024461 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.024453 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knv5w\" (UniqueName: \"kubernetes.io/projected/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-kube-api-access-knv5w\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:56:08.024461 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.024462 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/396ee576-3af2-4ddf-8b33-b4cc669f7b1b-tls-certs\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:56:08.649650 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.649614 2566 generic.go:356] "Generic (PLEG): container finished" podID="396ee576-3af2-4ddf-8b33-b4cc669f7b1b" containerID="c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b" exitCode=0 Jul 06 18:56:08.650143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.649664 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" Jul 06 18:56:08.650143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.649693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" event={"ID":"396ee576-3af2-4ddf-8b33-b4cc669f7b1b","Type":"ContainerDied","Data":"c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b"} Jul 06 18:56:08.650143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.649736 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp" event={"ID":"396ee576-3af2-4ddf-8b33-b4cc669f7b1b","Type":"ContainerDied","Data":"20a2896617be6ef84e547c91f4c4b68c4ff27e0acc336f9673205f6fce74ab90"} Jul 06 18:56:08.650143 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.649753 2566 scope.go:117] "RemoveContainer" containerID="c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b" Jul 06 18:56:08.659230 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.659212 2566 scope.go:117] "RemoveContainer" containerID="c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b" Jul 06 18:56:08.659528 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:56:08.659509 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b\": container with ID starting with c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b not found: ID does not exist" containerID="c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b" Jul 06 18:56:08.659587 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.659537 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b"} err="failed to get container status \"c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b\": rpc error: code = NotFound desc = could not find container \"c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b\": container with ID starting with c84ce78efd51ce00c7c0d7e2f5c25aaff302ff42fe01707aca680dc87154501b not found: ID does not exist" Jul 06 18:56:08.671965 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.671940 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp"] Jul 06 18:56:08.678184 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:08.678161 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/sub-test-model-b67177-kserve-6d49b5967f-2w7qp"] Jul 06 18:56:09.530716 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:09.530673 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="396ee576-3af2-4ddf-8b33-b4cc669f7b1b" path="/var/lib/kubelet/pods/396ee576-3af2-4ddf-8b33-b4cc669f7b1b/volumes" Jul 06 18:56:13.444071 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:13.444029 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:13.444515 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:13.444127 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:13.456679 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:13.456651 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:56:13.685894 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:56:13.685866 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:57:36.643118 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:36.643083 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872"] Jul 06 18:57:51.664901 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.664833 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" podUID="79ce691d-2172-4069-8ee0-8f1bbede5a89" containerName="main" containerID="cri-o://f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02" gracePeriod=45 Jul 06 18:57:51.904593 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.904567 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:57:51.945855 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.945770 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-tmp-dir\") pod \"79ce691d-2172-4069-8ee0-8f1bbede5a89\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " Jul 06 18:57:51.945855 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.945842 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ce691d-2172-4069-8ee0-8f1bbede5a89-tls-certs\") pod \"79ce691d-2172-4069-8ee0-8f1bbede5a89\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " Jul 06 18:57:51.946066 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.945873 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-dshm\") pod \"79ce691d-2172-4069-8ee0-8f1bbede5a89\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " Jul 06 18:57:51.946066 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.945902 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j57q5\" (UniqueName: \"kubernetes.io/projected/79ce691d-2172-4069-8ee0-8f1bbede5a89-kube-api-access-j57q5\") pod \"79ce691d-2172-4069-8ee0-8f1bbede5a89\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " Jul 06 18:57:51.946066 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.945917 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-model-cache\") pod \"79ce691d-2172-4069-8ee0-8f1bbede5a89\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " Jul 06 18:57:51.946066 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.945981 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-home\") pod \"79ce691d-2172-4069-8ee0-8f1bbede5a89\" (UID: \"79ce691d-2172-4069-8ee0-8f1bbede5a89\") " Jul 06 18:57:51.946066 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.946025 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79ce691d-2172-4069-8ee0-8f1bbede5a89" (UID: "79ce691d-2172-4069-8ee0-8f1bbede5a89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:51.946345 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.946259 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-tmp-dir\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:57:51.946345 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.946273 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-model-cache" (OuterVolumeSpecName: "model-cache") pod "79ce691d-2172-4069-8ee0-8f1bbede5a89" (UID: "79ce691d-2172-4069-8ee0-8f1bbede5a89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:51.946464 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.946437 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-home" (OuterVolumeSpecName: "home") pod "79ce691d-2172-4069-8ee0-8f1bbede5a89" (UID: "79ce691d-2172-4069-8ee0-8f1bbede5a89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:51.948624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.948595 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-dshm" (OuterVolumeSpecName: "dshm") pod "79ce691d-2172-4069-8ee0-8f1bbede5a89" (UID: "79ce691d-2172-4069-8ee0-8f1bbede5a89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:51.948624 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.948616 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ce691d-2172-4069-8ee0-8f1bbede5a89-kube-api-access-j57q5" (OuterVolumeSpecName: "kube-api-access-j57q5") pod "79ce691d-2172-4069-8ee0-8f1bbede5a89" (UID: "79ce691d-2172-4069-8ee0-8f1bbede5a89"). InnerVolumeSpecName "kube-api-access-j57q5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:57:51.948782 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:51.948615 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ce691d-2172-4069-8ee0-8f1bbede5a89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79ce691d-2172-4069-8ee0-8f1bbede5a89" (UID: "79ce691d-2172-4069-8ee0-8f1bbede5a89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:52.047257 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.047214 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ce691d-2172-4069-8ee0-8f1bbede5a89-tls-certs\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:57:52.047257 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.047246 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-dshm\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:57:52.047257 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.047256 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j57q5\" (UniqueName: \"kubernetes.io/projected/79ce691d-2172-4069-8ee0-8f1bbede5a89-kube-api-access-j57q5\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:57:52.047257 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.047265 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-model-cache\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:57:52.047257 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.047273 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ce691d-2172-4069-8ee0-8f1bbede5a89-home\") on node \"ip-10-0-133-250.ec2.internal\" DevicePath \"\"" Jul 06 18:57:52.152752 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.152718 2566 generic.go:356] "Generic (PLEG): container finished" podID="79ce691d-2172-4069-8ee0-8f1bbede5a89" containerID="f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02" exitCode=0 Jul 06 18:57:52.152942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.152758 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" Jul 06 18:57:52.152942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.152778 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" event={"ID":"79ce691d-2172-4069-8ee0-8f1bbede5a89","Type":"ContainerDied","Data":"f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02"} Jul 06 18:57:52.152942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.152817 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872" event={"ID":"79ce691d-2172-4069-8ee0-8f1bbede5a89","Type":"ContainerDied","Data":"975a2882f6949a55a58cb728e996f3fbbbc7d281662593e2056c50eb871c418a"} Jul 06 18:57:52.152942 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.152838 2566 scope.go:117] "RemoveContainer" containerID="f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02" Jul 06 18:57:52.162459 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.162372 2566 scope.go:117] "RemoveContainer" containerID="f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02" Jul 06 18:57:52.162684 ip-10-0-133-250 kubenswrapper[2566]: E0706 18:57:52.162667 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02\": container with ID starting with f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02 not found: ID does not exist" containerID="f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02" Jul 06 18:57:52.162725 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.162692 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02"} err="failed to get container status \"f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02\": rpc error: code = NotFound desc = could not find container \"f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02\": container with ID starting with f96f715c5ec567257c9f9353d56177ced4548cfe43bbf702d8bc51c80b641e02 not found: ID does not exist" Jul 06 18:57:52.177854 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.177829 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872"] Jul 06 18:57:52.180375 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:52.180355 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/rate-test-model-8c615a-kserve-f7c5d747-tl872"] Jul 06 18:57:53.530012 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:57:53.529971 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ce691d-2172-4069-8ee0-8f1bbede5a89" path="/var/lib/kubelet/pods/79ce691d-2172-4069-8ee0-8f1bbede5a89/volumes" Jul 06 18:58:34.812088 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:34.812057 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-ckrl4_9c5acd9d-4795-43be-842e-9df64881ddd3/copy-manifests/0.log" Jul 06 18:58:34.825700 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:34.825673 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-ckrl4_9c5acd9d-4795-43be-842e-9df64881ddd3/manager/0.log" Jul 06 18:58:36.839473 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:36.839444 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-r7m7s_377ee50b-a710-4a3b-b579-3a80d69cb41f/kuadrant-console-plugin/0.log" Jul 06 18:58:37.077190 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:37.077155 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8bszt_6ad0f006-50d1-4a0b-8815-d78ba411c169/manager/0.log" Jul 06 18:58:37.898950 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:37.898918 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-5pqvn_a59f1815-bef4-465d-93c7-e7e804c9f536/kube-auth-proxy/0.log" Jul 06 18:58:38.361816 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:38.361783 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-97vr9_9f04b1ab-0dcd-447c-9f31-5aa61200a792/payload-processing/0.log" Jul 06 18:58:39.047692 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:39.047660 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5n7mc_e36ad723-cfb9-44f7-b03e-9b6d0269c60c/main/0.log" Jul 06 18:58:39.163046 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:39.163016 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db82llg_8f6d2366-d3e7-438e-ba40-f6259b95f538/main/0.log" Jul 06 18:58:39.410559 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:39.410481 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vgjk8_1156af4a-b8ae-4c3d-b845-147530a24b49/main/0.log" Jul 06 18:58:46.394265 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:46.394236 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s6kxp_b429876b-7071-41ab-985a-a32f0b9ea8f4/global-pull-secret-syncer/0.log" Jul 06 18:58:46.604997 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:46.604967 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jfvv2_fc9e29be-1af8-419b-9847-e53db35a1c78/konnectivity-agent/0.log" Jul 06 18:58:46.670885 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:46.670796 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-250.ec2.internal_7c5d0c0b8e27dab8be106a1a4a27f027/haproxy/0.log" Jul 06 18:58:52.129090 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:52.129063 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-r7m7s_377ee50b-a710-4a3b-b579-3a80d69cb41f/kuadrant-console-plugin/0.log" Jul 06 18:58:52.318911 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:52.318877 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8bszt_6ad0f006-50d1-4a0b-8815-d78ba411c169/manager/0.log" Jul 06 18:58:54.074360 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.074270 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/alertmanager/0.log" Jul 06 18:58:54.100472 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.100440 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/config-reloader/0.log" Jul 06 18:58:54.124428 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.124404 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/kube-rbac-proxy-web/0.log" Jul 06 18:58:54.148455 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.148431 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/kube-rbac-proxy/0.log" Jul 06 18:58:54.172256 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.172227 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/kube-rbac-proxy-metric/0.log" Jul 06 18:58:54.196604 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.196579 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/prom-label-proxy/0.log" Jul 06 18:58:54.217308 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.217278 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c9b71495-bba2-4605-a416-1f886b718fc6/init-config-reloader/0.log" Jul 06 18:58:54.282517 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.282488 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tx8z2_4644d803-d556-4c44-9834-d963e0bade34/kube-state-metrics/0.log" Jul 06 18:58:54.300428 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.300403 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tx8z2_4644d803-d556-4c44-9834-d963e0bade34/kube-rbac-proxy-main/0.log" Jul 06 18:58:54.329083 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.329014 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tx8z2_4644d803-d556-4c44-9834-d963e0bade34/kube-rbac-proxy-self/0.log" Jul 06 18:58:54.361893 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.361871 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-554bdb969-pb47w_ac4d2a11-08ff-4906-9ae4-70f839284f5e/metrics-server/0.log" Jul 06 18:58:54.387261 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.387236 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-p8rkc_41455cdd-d221-4a01-91ff-3d5e428d17ad/monitoring-plugin/0.log" Jul 06 18:58:54.583724 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.583646 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zw9rt_b6e11408-82ee-4e85-a707-82dda8f4e968/node-exporter/0.log" Jul 06 18:58:54.606553 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.606525 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zw9rt_b6e11408-82ee-4e85-a707-82dda8f4e968/kube-rbac-proxy/0.log" Jul 06 18:58:54.634217 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.634194 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zw9rt_b6e11408-82ee-4e85-a707-82dda8f4e968/init-textfile/0.log" Jul 06 18:58:54.662752 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.662727 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sxjsk_36b5545f-2629-4e7e-9059-ec57ced9eefc/kube-rbac-proxy-main/0.log" Jul 06 18:58:54.685633 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.685609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sxjsk_36b5545f-2629-4e7e-9059-ec57ced9eefc/kube-rbac-proxy-self/0.log" Jul 06 18:58:54.706952 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.706926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sxjsk_36b5545f-2629-4e7e-9059-ec57ced9eefc/openshift-state-metrics/0.log" Jul 06 18:58:54.740696 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.740671 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/prometheus/0.log" Jul 06 18:58:54.760716 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.760693 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/config-reloader/0.log" Jul 06 18:58:54.781816 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.781778 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/thanos-sidecar/0.log" Jul 06 18:58:54.803267 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.803237 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/kube-rbac-proxy-web/0.log" Jul 06 18:58:54.826868 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.826840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/kube-rbac-proxy/0.log" Jul 06 18:58:54.847512 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.847439 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/kube-rbac-proxy-thanos/0.log" Jul 06 18:58:54.867682 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.867657 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa9a54c-39df-4941-90c8-d74f8301e887/init-config-reloader/0.log" Jul 06 18:58:54.897890 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.897865 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-znq4p_a5e751c7-9dfe-422f-8d91-225bf70c29ec/prometheus-operator/0.log" Jul 06 18:58:54.916391 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.916363 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-znq4p_a5e751c7-9dfe-422f-8d91-225bf70c29ec/kube-rbac-proxy/0.log" Jul 06 18:58:54.952255 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.952229 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-rjsgm_6c533e26-ef80-448c-ad43-0e78fae2167a/prometheus-operator-admission-webhook/0.log" Jul 06 18:58:54.980998 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:54.980973 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c8c949d8d-r4cxc_1569f168-17f2-401a-b032-5dac65c6e8ce/telemeter-client/0.log" Jul 06 18:58:55.007240 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.007212 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c8c949d8d-r4cxc_1569f168-17f2-401a-b032-5dac65c6e8ce/reload/0.log" Jul 06 18:58:55.037445 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.037422 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c8c949d8d-r4cxc_1569f168-17f2-401a-b032-5dac65c6e8ce/kube-rbac-proxy/0.log" Jul 06 18:58:55.067504 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.067480 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/thanos-query/0.log" Jul 06 18:58:55.087167 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.087146 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy-web/0.log" Jul 06 18:58:55.108295 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.108223 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy/0.log" Jul 06 18:58:55.132990 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.132967 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/prom-label-proxy/0.log" Jul 06 18:58:55.158071 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.158046 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy-rules/0.log" Jul 06 18:58:55.180631 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:55.180609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7748d77ff4-x8j2b_4510d5ec-0a37-4718-b3dc-b24cb8322e81/kube-rbac-proxy-metrics/0.log" Jul 06 18:58:56.110350 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110302 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd"] Jul 06 18:58:56.110757 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110703 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="396ee576-3af2-4ddf-8b33-b4cc669f7b1b" containerName="main" Jul 06 18:58:56.110757 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110716 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="396ee576-3af2-4ddf-8b33-b4cc669f7b1b" containerName="main" Jul 06 18:58:56.110757 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110736 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ce691d-2172-4069-8ee0-8f1bbede5a89" containerName="main" Jul 06 18:58:56.110757 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110742 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ce691d-2172-4069-8ee0-8f1bbede5a89" containerName="main" Jul 06 18:58:56.110912 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110798 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="396ee576-3af2-4ddf-8b33-b4cc669f7b1b" containerName="main" Jul 06 18:58:56.110912 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.110806 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ce691d-2172-4069-8ee0-8f1bbede5a89" containerName="main" Jul 06 18:58:56.113759 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.113735 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.116274 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.116254 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xw5b\"/\"kube-root-ca.crt\"" Jul 06 18:58:56.116424 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.116296 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xw5b\"/\"openshift-service-ca.crt\"" Jul 06 18:58:56.116424 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.116300 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8xw5b\"/\"default-dockercfg-xdgdk\"" Jul 06 18:58:56.122657 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.122637 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd"] Jul 06 18:58:56.256911 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.256871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-podres\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.257092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.256934 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-lib-modules\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.257092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.256976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-sys\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.257092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.257009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-proc\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.257092 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.257036 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw52c\" (UniqueName: \"kubernetes.io/projected/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-kube-api-access-xw52c\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357518 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357483 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-podres\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357685 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-lib-modules\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357685 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357566 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-sys\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357685 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-proc\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357685 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw52c\" (UniqueName: \"kubernetes.io/projected/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-kube-api-access-xw52c\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357685 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357655 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-podres\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357866 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357691 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-sys\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357866 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357708 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-proc\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.357866 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.357726 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-lib-modules\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.366570 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.366516 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw52c\" (UniqueName: \"kubernetes.io/projected/2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512-kube-api-access-xw52c\") pod \"perf-node-gather-daemonset-r2sbd\" (UID: \"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.425496 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.425457 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:56.563289 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:56.563265 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd"] Jul 06 18:58:56.565499 ip-10-0-133-250 kubenswrapper[2566]: W0706 18:58:56.565470 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2ad8dc8b_bfa4_4ed5_90cf_bcbcd7a25512.slice/crio-d212a8766aff96de3d04b7063f50a4b2b9258475d0055f32ac5e05fe43137bb6 WatchSource:0}: Error finding container d212a8766aff96de3d04b7063f50a4b2b9258475d0055f32ac5e05fe43137bb6: Status 404 returned error can't find the container with id d212a8766aff96de3d04b7063f50a4b2b9258475d0055f32ac5e05fe43137bb6 Jul 06 18:58:57.467591 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:57.467553 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" event={"ID":"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512","Type":"ContainerStarted","Data":"7a4e33bcedbb580faf40cd24b547d62244bf9c2265dab65b3cc2aad988970f1a"} Jul 06 18:58:57.468006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:57.467608 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" event={"ID":"2ad8dc8b-bfa4-4ed5-90cf-bcbcd7a25512","Type":"ContainerStarted","Data":"d212a8766aff96de3d04b7063f50a4b2b9258475d0055f32ac5e05fe43137bb6"} Jul 06 18:58:57.468006 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:57.467657 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:58:57.484393 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:57.484348 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" podStartSLOduration=1.4843341319999999 podStartE2EDuration="1.484334132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:58:57.483370466 +0000 UTC m=+2468.629911387" watchObservedRunningTime="2026-07-06 18:58:57.484334132 +0000 UTC m=+2468.630875045" Jul 06 18:58:59.228003 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:59.227978 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hm4wx_c1e3dc77-a500-45c4-81d1-9c27192aeff4/dns/0.log" Jul 06 18:58:59.248341 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:59.248299 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hm4wx_c1e3dc77-a500-45c4-81d1-9c27192aeff4/kube-rbac-proxy/0.log" Jul 06 18:58:59.385001 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:59.384969 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xqglp_8d9b7540-f14d-4f63-b8af-3afda5aad1c4/dns-node-resolver/0.log" Jul 06 18:58:59.955419 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:58:59.955394 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ccpbk_0de316fb-6637-4086-8cb1-e077df4a27b5/node-ca/0.log" Jul 06 18:59:01.136115 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:01.136087 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-5pqvn_a59f1815-bef4-465d-93c7-e7e804c9f536/kube-auth-proxy/0.log" Jul 06 18:59:01.344704 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:01.344670 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-97vr9_9f04b1ab-0dcd-447c-9f31-5aa61200a792/payload-processing/0.log" Jul 06 18:59:02.101863 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:02.101835 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-twqff_c4854bf1-bb57-4d1f-a03d-2277f52564ff/serve-healthcheck-canary/0.log" Jul 06 18:59:02.650719 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:02.650688 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rkvwj_50f70ab8-b258-480a-bc7a-eb2315a4666c/kube-rbac-proxy/0.log" Jul 06 18:59:02.679294 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:02.679267 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rkvwj_50f70ab8-b258-480a-bc7a-eb2315a4666c/exporter/0.log" Jul 06 18:59:02.701291 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:02.701261 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rkvwj_50f70ab8-b258-480a-bc7a-eb2315a4666c/extractor/0.log" Jul 06 18:59:03.482542 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:03.482517 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-r2sbd" Jul 06 18:59:05.590651 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:05.590611 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-ckrl4_9c5acd9d-4795-43be-842e-9df64881ddd3/manager/0.log" Jul 06 18:59:05.608172 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:05.608147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-ckrl4_9c5acd9d-4795-43be-842e-9df64881ddd3/copy-manifests/0.log" Jul 06 18:59:07.142160 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:07.142135 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c45f78566-mbnmp_dc1501ca-b812-42aa-9433-053ba95dd289/manager/0.log" Jul 06 18:59:07.172101 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:07.172075 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qngqb_7865cd18-4584-4c20-b3e2-6081251e5269/openshift-lws-operator/0.log" Jul 06 18:59:07.366235 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:07.366150 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-5pqvn_a59f1815-bef4-465d-93c7-e7e804c9f536/kube-auth-proxy/0.log" Jul 06 18:59:07.510583 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:07.510548 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-97vr9_9f04b1ab-0dcd-447c-9f31-5aa61200a792/payload-processing/0.log" Jul 06 18:59:08.838417 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:08.838393 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-5pqvn_a59f1815-bef4-465d-93c7-e7e804c9f536/kube-auth-proxy/0.log" Jul 06 18:59:09.013631 ip-10-0-133-250 kubenswrapper[2566]: I0706 18:59:09.013601 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-97vr9_9f04b1ab-0dcd-447c-9f31-5aa61200a792/payload-processing/0.log"