Jul 02 19:03:10.502579 ip-10-0-133-120 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 02 19:03:10.502589 ip-10-0-133-120 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 02 19:03:10.502596 ip-10-0-133-120 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 02 19:03:10.502810 ip-10-0-133-120 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 02 19:03:20.670448 ip-10-0-133-120 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 02 19:03:20.670463 ip-10-0-133-120 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 91508dbe34184ec4916d1b9ea2c8c32b --
Jul 02 19:05:17.666878 ip-10-0-133-120 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 19:05:17.961407 ip-10-0-133-120 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:05:17.961407 ip-10-0-133-120 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 19:05:17.961407 ip-10-0-133-120 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:05:17.961407 ip-10-0-133-120 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 19:05:17.961407 ip-10-0-133-120 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:05:17.962223 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.962052    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 19:05:17.965872 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965858    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:05:17.965872 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965872    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965876    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965879    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965882    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965885    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965888    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965891    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965894    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965897    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965900    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965903    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965907    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965910    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965913    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965916    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965920    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965924    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965927    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965930    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:05:17.965940 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965933    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965936    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965940    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965944    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965948    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965950    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965954    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965957    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965960    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965963    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965965    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965968    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965971    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965973    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965976    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965978    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965981    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965983    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965986    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965988    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:05:17.966429 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965991    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965993    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965996    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.965999    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966001    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966004    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966007    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966010    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966012    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966014    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966017    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966020    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966022    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966025    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966028    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966030    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966033    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966035    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966038    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:05:17.966933 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966040    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966043    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966045    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966048    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966050    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966053    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966055    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966058    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966060    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966063    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966065    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966068    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966071    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966073    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966077    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966079    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966082    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966085    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966087    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966090    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:05:17.967404 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966092    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966095    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966097    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966100    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966102    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966105    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966108    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966111    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966114    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966116    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966118    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966121    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966124    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966126    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966129    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966131    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966133    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966136    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966139    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:05:17.967891 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.966142    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:05:17.968466 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968453    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:05:17.968466 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968465    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968468    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968473    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968476    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968480    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968484    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968486    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968489    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968492    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968495    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968498    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968500    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968503    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968506    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968508    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968511    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968513    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968516    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968519    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:05:17.968515 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968522    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968524    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968527    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968530    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968533    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968535    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968538    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968540    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968543    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968545    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968547    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968550    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968553    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968555    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968558    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968560    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968563    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968565    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968568    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968570    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:05:17.969028 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968573    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968575    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968578    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968580    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968583    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968585    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968600    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968603    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968606    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968608    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968611    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968613    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968616    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968618    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968622    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968625    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968628    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968630    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968633    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968636    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:05:17.969528 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968638    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968641    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968643    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968646    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968648    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968651    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968654    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968656    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968659    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968661    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968664    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968667    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968670    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968673    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968675    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968678    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968680    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968683    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968685    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968688    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:05:17.970053 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968692    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968695    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968697    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968701    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968705    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968708    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968712    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968715    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968717    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968720    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968722    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968725    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968729    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968732    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968735    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968738    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968741    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968744    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.968747    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:05:17.970535 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969113    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969117    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969120    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969124    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969126    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969129    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969131    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969134    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969137    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969139    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969143    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969147    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969149    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969152    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969155    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969158    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969161    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969164    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969167    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969170    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:05:17.971027 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969173    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969176    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969178    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969181    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969185    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969188    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969191    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969194    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969196    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969199    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969203    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969206    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969209    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969211    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969214    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969217    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969219    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969222    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969224    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:05:17.971505 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969227    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969229    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969232    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969235    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969238    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969240    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969243    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969245    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969248    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969250    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969253    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969256    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969259    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969261    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969265    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969268    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969270    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969273    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969275    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969278    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:05:17.971963 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969280    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969283    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969285    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969288    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969290    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969293    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969295    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969298    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969300    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969303    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969306    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969308    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969311    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969313    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969316    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969318    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969320    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969323    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969325    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:05:17.972450 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969328    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969330    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969333    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969336    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969339    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969341    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969344    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969346    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969349    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969351    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969354    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969356    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969359    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969361    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969364    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969366    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969369    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969371    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969374    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969376    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.969379    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:05:17.972932 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969450    2585 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969460    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969467    2585 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969472    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969476    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969479    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969483    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969487    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969493    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969496    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969501    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969504    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969507    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969510    2585 flags.go:64] FLAG: --cgroup-root=""
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969513    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969516    2585 flags.go:64] FLAG: --client-ca-file=""
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969519    2585 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969522    2585 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969526    2585 flags.go:64] FLAG: --cluster-domain=""
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969529    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969533    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969536    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969539    2585 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969543    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 19:05:17.973448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969546    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969549    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969553    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969556    2585 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969559    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969562    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969565    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969569    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969573    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969576    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969579    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969582    2585 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969585    2585 flags.go:64] FLAG: --enable-server="true"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969601    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969607    2585 flags.go:64] FLAG: --event-burst="100"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969610    2585 flags.go:64] FLAG: --event-qps="50"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969615    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969618    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969621    2585 flags.go:64] FLAG: --eviction-hard=""
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969625    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969628    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969631    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969634    2585 flags.go:64] FLAG: --eviction-soft=""
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969637    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969640    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 19:05:17.974060 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969643    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969646    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969649    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969652    2585 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969655    2585 flags.go:64] FLAG: --feature-gates=""
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969659    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969662    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969665    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969668    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969671    2585 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969674    2585 flags.go:64] FLAG: --help="false"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969677    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-133-120.ec2.internal"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969680    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969683    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969686    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969689    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969692    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969695    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969698    2585 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969702    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969705    2585 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969708    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969711    2585 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969715    2585 flags.go:64] FLAG: --kube-reserved=""
Jul 02 19:05:17.974691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969718    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969722    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969725    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969728    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969731    2585 flags.go:64] FLAG: --lock-file=""
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969733    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969736    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969739    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969744    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969747    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969750    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969753    2585 flags.go:64] FLAG: --logging-format="text"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969756    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969759    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969762    2585 flags.go:64] FLAG: --manifest-url=""
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969765    2585 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969769    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969772    2585 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969776    2585 flags.go:64] FLAG: --max-pods="110"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969779    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969782    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969784    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969787    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969790    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969793    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 19:05:17.975272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969796    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969803    2585 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969806    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969809    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969812    2585 flags.go:64] FLAG: --pod-cidr=""
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969815    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969821    2585 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969825    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969828    2585 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969832    2585 flags.go:64] FLAG: --port="10250"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969835    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969838    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d3d5a46015686c01"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969841    2585 flags.go:64] FLAG: --qos-reserved=""
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969844    2585 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969847    2585 flags.go:64] FLAG: --register-node="true"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969850    2585 flags.go:64] FLAG: --register-with-taints=""
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969853    2585 flags.go:64] FLAG: --registry-burst="10"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969856    2585 flags.go:64] FLAG: --registry-qps="5"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969859    2585 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969861    2585 flags.go:64] FLAG: --reserved-memory=""
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969865    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969868    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969871    2585 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969873    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969876    2585 flags.go:64] FLAG: --runonce="false"
Jul 02 19:05:17.975946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969879    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969883    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969886    2585 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969888    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969891    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969894    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969897    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969900    2585 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969903    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969906    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969909    2585 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969912    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969916    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969919    2585 flags.go:64] FLAG: --system-cgroups=""
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969925    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969931    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969934    2585 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969937    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969941    2585 flags.go:64] FLAG: --tls-min-version=""
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969944    2585 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969947    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969950    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969952    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969955    2585 flags.go:64] FLAG: --v="2"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969959    2585 flags.go:64] FLAG: --version="false"
Jul 02 19:05:17.976561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969963    2585 flags.go:64] FLAG: --vmodule=""
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969966    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.969970    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970090    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970093    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970096    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970100    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970104    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970107    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970110    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970114    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970117    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970122    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970125    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970128    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970131    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970133    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970136    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970139    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:05:17.977402 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970148    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970151    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970154    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970157    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970159    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970162    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970165    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970167    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970170    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970173    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970175    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970178    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970180    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970183    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970185    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970188    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970190    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970193    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970195    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970198    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:05:17.978168 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970201    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970203    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970206    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970208    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970211    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970215    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970218    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970220    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970223    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970226    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970228    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970231    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970234    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970241    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970246    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970248    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970251    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970254    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970256    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970259    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:05:17.979066 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970261    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970264    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970266    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970269    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970271    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970274    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970277    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970279    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970281    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970284    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970287    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970289    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970291    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970294    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970297    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970299    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970302    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970306    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970308    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970311    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:05:17.979876 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970313    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970316    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970318    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970321    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970323    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970325    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970330    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970332    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970335    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970338    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970340    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970343    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970345    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970348    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970351    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970353    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970356    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970358    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970361    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970363    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:05:17.980661 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970366    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970368    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.970372    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.970379    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.977126    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.977211    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.977798    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.977813    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978252    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978265    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978271    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978276    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978281    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978286    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:05:17.981192 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978291    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978296    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978301    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978305    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978310    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978314    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978319    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978322    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978327    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978331    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978335    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978340    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978344    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978348    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978353    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978357    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978361    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978366    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978370    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978374    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:05:17.981728 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978379    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978383    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978388    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978393    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978400    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978406    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978411    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978417    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978423    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978427    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978432    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978437    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978441    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978447    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978451    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978456    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978460    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978464    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978469    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:05:17.982499 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978473    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978477    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978481    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978485    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978489    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978494    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978498    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978502    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978506    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978511    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978515    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978519    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978524    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978528    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978534    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978540    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978546    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978550    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978554    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978558    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:05:17.983208 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978564    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978580    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978584    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978604    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978608    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978612    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978615    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978620    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978624    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978630    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978633    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978638    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978642    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978646    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978650    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978654    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978658    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978662    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978667    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978671    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:05:17.983760 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978675    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978679    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978683    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978687    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978692    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978696    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978700    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978704    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978708    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978715    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978721    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978726    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978730    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978735    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:05:17.984308 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.978743    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978957    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978967    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978974    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978979    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978983    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978987    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978992    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.978996    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979000    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979007    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979013    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979017    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979030    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979036    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979040    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979044    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979049    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979053    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979057    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:05:17.984675 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979062    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979066    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979070    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979074    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979078    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979082    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979087    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979092    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979097    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979102    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979107    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979112    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979116    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979121    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979125    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979129    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979133    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979137    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979142    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:05:17.985229 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979146    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979150    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979154    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979159    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979163    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979167    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979171    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979185    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979190    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979195    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979199    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979203    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979207    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979211    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979216    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979220    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979224    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979228    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979232    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979237    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:05:17.985739 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979241    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979245    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979249    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979254    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979258    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979262    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979266    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979271    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979275    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979280    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979284    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979288    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979292    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979296    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979301    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979305    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979309    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979313    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979317    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979321    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:05:17.986577 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979333    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979338    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979342    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979346    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979350    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979354    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979358    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979362    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979367    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979371    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979375    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979379    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979383    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979388    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979393    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979397    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979402    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979407    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979411    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:05:17.987413 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979415    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:17.979419    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.979428    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.979628    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.982920    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.984096    2585 server.go:1019] "Starting client certificate rotation"
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.984203    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.984235    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 19:05:17.988230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.986415    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 19:05:17.988526 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.988373    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 19:05:17.994268 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.994250    2585 log.go:25] "Validated CRI v1 runtime API"
Jul 02 19:05:17.996245 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.996232    2585 log.go:25] "Validated CRI v1 image API"
Jul 02 19:05:17.997042 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:17.997024    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 19:05:18.000422 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.000400    2585 fs.go:135] Filesystem UUIDs: map[3bdde753-6929-4f4f-b760-92d76f264ada:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c69a2cae-8b40-4451-b231-413a907c37ef:/dev/nvme0n1p3]
Jul 02 19:05:18.000485 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.000424    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 19:05:18.005577 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.005470    2585 manager.go:216] Machine: {Timestamp:2026-07-02 19:05:18.004463281 +0000 UTC m=+0.247052638 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095925 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec243fd2f9d8951a1d64cf5d7922f514 SystemUUID:ec243fd2-f9d8-951a-1d64-cf5d7922f514 BootID:91508dbe-3418-4ec4-916d-1b9ea2c8c32b Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:92:7f:d9:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:92:7f:d9:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:a1:53:5a:07:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 19:05:18.005577 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.005571    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 19:05:18.005681 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.005665    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 19:05:18.006489 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.006467    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 19:05:18.006657 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.006491    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 19:05:18.007132 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.007122    2585 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 19:05:18.007169 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.007134    2585 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 19:05:18.007169 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.007142    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 19:05:18.008183 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.008169    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 19:05:18.009347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.009338    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 19:05:18.009709 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.009692    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 19:05:18.013079 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013068    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 19:05:18.013190 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013181    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 19:05:18.013228 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013210    2585 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 19:05:18.013228 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013219    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 19:05:18.013281 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013235    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 19:05:18.013281 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013247    2585 kubelet.go:398] "Adding apiserver pod source"
Jul 02 19:05:18.013281 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013258    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 19:05:18.013732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013714    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 19:05:18.013817 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.013744    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 19:05:18.017978 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.017957    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 19:05:18.019645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.019631    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 19:05:18.019712 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.019661    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 19:05:18.020547 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020536    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 19:05:18.020579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020554    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 19:05:18.020579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020565    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 19:05:18.020579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020573    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020582    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020603    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020614    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020619    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020625    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020631    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020638    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 19:05:18.020688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.020644    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 19:05:18.021267 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.021256    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 19:05:18.021295 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.021270    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 19:05:18.023620 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.023576    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 19:05:18.023696 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.023628    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 19:05:18.023696 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.023667    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 19:05:18.024498 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.024486    2585 server.go:1268] "Started kubelet"
Jul 02 19:05:18.024585 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.024547    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 19:05:18.024681 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.024561    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 19:05:18.024760 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.024685    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 19:05:18.025353 ip-10-0-133-120 systemd[1]: Started Kubernetes Kubelet.
Jul 02 19:05:18.026321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.026179    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 19:05:18.026427 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.026414    2585 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 19:05:18.028911 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.028893    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 19:05:18.031579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.031565    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 19:05:18.031722 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.031696    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:18.031824 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.031802    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 19:05:18.032105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.032087    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 19:05:18.032294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.032281    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 19:05:18.032340 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.032298    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 19:05:18.032399 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.032388    2585 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 19:05:18.032434 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.032399    2585 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 19:05:18.034684 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.034665    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 02 19:05:18.034911 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.034533    2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 02 19:05:18.035020 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.034956    2585 factory.go:55] Registering systemd factory
Jul 02 19:05:18.035020 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.034969    2585 factory.go:223] Registration of the systemd container factory successfully
Jul 02 19:05:18.035209 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.034320    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e5940613  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.024459795 +0000 UTC m=+0.267049153,LastTimestamp:2026-07-02 19:05:18.024459795 +0000 UTC m=+0.267049153,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.035281 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.035227    2585 factory.go:153] Registering CRI-O factory
Jul 02 19:05:18.035281 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.035236    2585 factory.go:223] Registration of the crio container factory successfully
Jul 02 19:05:18.035345 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.035288    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 19:05:18.035345 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.035317    2585 factory.go:103] Registering Raw factory
Jul 02 19:05:18.035345 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.035334    2585 manager.go:1196] Started watching for new ooms in manager
Jul 02 19:05:18.035955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.035943    2585 manager.go:349] Starting recovery of all containers
Jul 02 19:05:18.047639 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.047623    2585 manager.go:354] Recovery completed
Jul 02 19:05:18.051670 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.051655    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:18.053411 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.053387    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:18.053467 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.053429    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:18.053467 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.053441    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:18.053996 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.053982    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 19:05:18.053996 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.053995    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 19:05:18.054090 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.053987    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:18.054090 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.054013    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 19:05:18.054659 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.054644    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:18.054709 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.054669    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:18.054709 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.054682    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:18.056195 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.056184    2585 policy_none.go:49] "None policy: Start"
Jul 02 19:05:18.056236 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.056199    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 19:05:18.056236 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.056209    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 19:05:18.058093 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.058083    2585 policy_none.go:47] "Start"
Jul 02 19:05:18.065024 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.064864    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e74dcff3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.053412851 +0000 UTC m=+0.296002216,LastTimestamp:2026-07-02 19:05:18.053412851 +0000 UTC m=+0.296002216,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.067881 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.067822    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e74e262b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.053434923 +0000 UTC m=+0.296024279,LastTimestamp:2026-07-02 19:05:18.053434923 +0000 UTC m=+0.296024279,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.078476 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.078418    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e74e4d27  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.053444903 +0000 UTC m=+0.296034259,LastTimestamp:2026-07-02 19:05:18.053444903 +0000 UTC m=+0.296034259,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.081895 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.081836    2585 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-120.ec2.internal.18be8e70e74dcff3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e74dcff3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.053412851 +0000 UTC m=+0.296002216,LastTimestamp:2026-07-02 19:05:18.05465635 +0000 UTC m=+0.297245706,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.090441 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.090371    2585 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-120.ec2.internal.18be8e70e74e262b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e74e262b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.053434923 +0000 UTC m=+0.296024279,LastTimestamp:2026-07-02 19:05:18.054677417 +0000 UTC m=+0.297266773,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.091228 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091211    2585 manager.go:337] "Starting Device Plugin manager"
Jul 02 19:05:18.091314 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.091248    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 19:05:18.091314 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091261    2585 server.go:87] "Starting device plugin registration server"
Jul 02 19:05:18.091510 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091500    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 19:05:18.091543 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091512    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 19:05:18.091656 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091635    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 19:05:18.091738 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091728    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 19:05:18.091738 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.091738    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 19:05:18.092179 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.092162    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 19:05:18.092231 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.092219    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:18.101218 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.101151    2585 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-120.ec2.internal.18be8e70e74e4d27\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e74e4d27  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.053444903 +0000 UTC m=+0.296034259,LastTimestamp:2026-07-02 19:05:18.054686534 +0000 UTC m=+0.297275891,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.108861 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.108844    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xjqt2"
Jul 02 19:05:18.110412 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:18.110351    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18be8e70e9b16a2e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-07-02 19:05:18.09349483 +0000 UTC m=+0.336084176,LastTimestamp:2026-07-02 19:05:18.09349483 +0000 UTC m=+0.336084176,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Jul 02 19:05:18.119333 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:18.119315    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xjqt2"
Jul 02 19:05:19.284414 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.284365    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:19.285446 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.285427    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:19.285551 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.285463    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:19.285551 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.285479    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:19.285551 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.285512    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.301019 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.300999    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.301126 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.301024    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-120.ec2.internal\": node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.307604 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.307532    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 19:05:19.308681 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.308668    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 19:05:19.308754 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.308697    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 19:05:19.308754 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.308722    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 19:05:19.308754 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.308751    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 02 19:05:19.310913 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.310894    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:05:19.315437 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.315418    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.408991 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.408952    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"]
Jul 02 19:05:19.409124 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.409026    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:19.409870 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.409851    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:19.409958 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.409886    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:19.409958 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.409901    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:19.412308 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.412295    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:19.412414 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.412400    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.412447 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.412430    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:19.412985 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.412967    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:19.413077 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.412997    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:19.413077 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.413007    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:19.413077 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.412967    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:19.413202 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.413079    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:19.413202 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.413091    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:19.415427 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.415412    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.415505 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.415436    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:19.415505 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.415469    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.416023 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.416005    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:19.416099 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.416033    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:19.416099 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.416046    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:19.426547 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.426532    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91bf6946301bf34241dbf61205b71188-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"91bf6946301bf34241dbf61205b71188\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.426634 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.426554    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91bf6946301bf34241dbf61205b71188-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"91bf6946301bf34241dbf61205b71188\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.426634 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.426570    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ad4923ba204444f53390e622f229eba-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"4ad4923ba204444f53390e622f229eba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.432657 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.432638    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-120.ec2.internal\" not found" node="ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.435988 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.435973    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-120.ec2.internal\" not found" node="ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.515609 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.515589    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.527233 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.527216    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91bf6946301bf34241dbf61205b71188-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"91bf6946301bf34241dbf61205b71188\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.527321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.527241    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91bf6946301bf34241dbf61205b71188-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"91bf6946301bf34241dbf61205b71188\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.527321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.527258    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ad4923ba204444f53390e622f229eba-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"4ad4923ba204444f53390e622f229eba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.527321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.527304    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ad4923ba204444f53390e622f229eba-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"4ad4923ba204444f53390e622f229eba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.527456 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.527317    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91bf6946301bf34241dbf61205b71188-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"91bf6946301bf34241dbf61205b71188\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.527456 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.527338    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91bf6946301bf34241dbf61205b71188-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"91bf6946301bf34241dbf61205b71188\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.616516 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.616434    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.717018 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.716970    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.735421 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.735401    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.738733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:19.738716    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:19.817462 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.817424    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:19.918014 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:19.917973    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:20.018475 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:20.018435    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:20.076970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.076946    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 19:05:20.077415 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.077079    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 19:05:20.119281 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:20.119259    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:20.170042 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.169987    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:05:20.176073 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.176049    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:05:20.177775 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.177757    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:05:20.177887 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.177790    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:05:20.177887 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.177801    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:05:20.215511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.215464    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 19:00:19 +0000 UTC" deadline="2028-01-16 01:57:38.402693437 +0000 UTC"
Jul 02 19:05:20.215511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.215508    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13494h52m18.187189388s"
Jul 02 19:05:20.219592 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:20.219571    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:20.281243 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:20.281210    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91bf6946301bf34241dbf61205b71188.slice/crio-bbd0e00d2f445c89ba729ad8469bd56ddef8bf4fd8cdeb30a5238e9f2e9445e2 WatchSource:0}: Error finding container bbd0e00d2f445c89ba729ad8469bd56ddef8bf4fd8cdeb30a5238e9f2e9445e2: Status 404 returned error can't find the container with id bbd0e00d2f445c89ba729ad8469bd56ddef8bf4fd8cdeb30a5238e9f2e9445e2
Jul 02 19:05:20.286525 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.286511    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:05:20.307293 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:20.307270    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad4923ba204444f53390e622f229eba.slice/crio-2b9e5e377c8a15008d6108750b1eaae80f5c52679960f4ee4cdbb35ad7e08535 WatchSource:0}: Error finding container 2b9e5e377c8a15008d6108750b1eaae80f5c52679960f4ee4cdbb35ad7e08535: Status 404 returned error can't find the container with id 2b9e5e377c8a15008d6108750b1eaae80f5c52679960f4ee4cdbb35ad7e08535
Jul 02 19:05:20.311881 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.311847    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" event={"ID":"4ad4923ba204444f53390e622f229eba","Type":"ContainerStarted","Data":"2b9e5e377c8a15008d6108750b1eaae80f5c52679960f4ee4cdbb35ad7e08535"}
Jul 02 19:05:20.312776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.312756    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"91bf6946301bf34241dbf61205b71188","Type":"ContainerStarted","Data":"bbd0e00d2f445c89ba729ad8469bd56ddef8bf4fd8cdeb30a5238e9f2e9445e2"}
Jul 02 19:05:20.320357 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:20.320339    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:20.348195 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.348176    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:05:20.420717 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:20.420658    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Jul 02 19:05:20.456897 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.456878    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:05:20.525120 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.525100    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:20.537196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.537180    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 19:05:20.537813 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.537802    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Jul 02 19:05:20.544625 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:20.544610    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 19:05:21.106854 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.106579    2585 apiserver.go:52] "Watching apiserver"
Jul 02 19:05:21.115958 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.115930    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 19:05:21.116444 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.116408    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-xztmn","openshift-image-registry/node-ca-kjg2x","openshift-multus/multus-7hw4t","openshift-multus/multus-additional-cni-plugins-dft29","openshift-multus/network-metrics-daemon-crgh4","kube-system/konnectivity-agent-c6czg","kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal","openshift-network-diagnostics/network-check-target-vrgc9","openshift-network-operator/iptables-alerter-2mzmt","openshift-ovn-kubernetes/ovnkube-node-sgwpk","kube-system/global-pull-secret-syncer-fc92r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8","openshift-cluster-node-tuning-operator/tuned-v2j5n"]
Jul 02 19:05:21.119309 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.119290    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.121384 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.121235    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.122190 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.122171    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5twqp\""
Jul 02 19:05:21.122718 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.122462    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.122718 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.122534    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.122718 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.122702    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 19:05:21.123581 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.123410    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.123581 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.123520    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.124263 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.123983    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 19:05:21.124263 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.124113    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.124263 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.124211    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pv6g2\""
Jul 02 19:05:21.125401 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.125385    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 19:05:21.125776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.125754    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.126322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.126302    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 19:05:21.126806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.126788    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t9xkw\""
Jul 02 19:05:21.126806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.126798    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 19:05:21.127418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.127304    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.127519 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.127418    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.128881 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.128529    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zrvzn\""
Jul 02 19:05:21.129664 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.129644    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 19:05:21.129893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.129875    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 19:05:21.130272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.130253    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.130396 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.130373    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:21.132987 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.132967    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.135135 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135104    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:21.135290 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.135267    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:21.135376 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135355    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-system-cni-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135426 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135391    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-etc-selinux\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.135426 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135415    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-daemon-config\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135525 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-registration-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.135525 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-os-release\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135525 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135521    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/473b186a-061e-4255-9d89-ed1f0e875452-host-slash\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.135672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135546    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kvwj\" (UniqueName: \"kubernetes.io/projected/473b186a-061e-4255-9d89-ed1f0e875452-kube-api-access-2kvwj\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.135672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135571    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-device-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.135672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135595    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-cni-binary-copy\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.135672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135619    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-socket-dir-parent\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135642    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-cni-bin\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135664    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-cni-multus\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-kubelet\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135712    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-conf-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135734    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/473b186a-061e-4255-9d89-ed1f0e875452-iptables-alerter-script\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135759    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d31aa88e-4093-4102-915e-ccd7cbd19178-cni-binary-copy\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135780    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-netns\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135810    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-cni-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135833    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-k8s-cni-cncf-io\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135857    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-cnibin\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.135949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135880    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.135882    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136005    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cjxv\" (UniqueName: \"kubernetes.io/projected/e9336630-072c-4de9-af56-044b1f64ecef-kube-api-access-2cjxv\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136070    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-socket-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136089    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136093    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22hfb\" (UniqueName: \"kubernetes.io/projected/5fdc6368-8b21-43d3-9809-4c219ad9f06d-kube-api-access-22hfb\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136118    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-etc-kubernetes\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136141    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8xp6\" (UniqueName: \"kubernetes.io/projected/d31aa88e-4093-4102-915e-ccd7cbd19178-kube-api-access-d8xp6\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136171    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w65xh\""
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136185    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-system-cni-dir\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136207    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-os-release\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136284    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136320    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136364    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136388    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-cnibin\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.136475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136459    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-hostroot\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.136976 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-multus-certs\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.136976 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.136510    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-sys-fs\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.137645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.137613    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.139660 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.139642    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.140492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.140416    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v25b2\""
Jul 02 19:05:21.140698 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.140645    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.140746 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.140647    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.140931 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.140915    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 19:05:21.141983 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.141962    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.142365 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.142174    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.142365 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.142275    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rzc27\""
Jul 02 19:05:21.142619 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.142579    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 19:05:21.143109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.143062    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.145496 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.145420    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.145712 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.145696    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.145809 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.145793    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g5jlc\""
Jul 02 19:05:21.145872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.145852    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.148112 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.148089    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.148252 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.148168    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:21.148252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.148189    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 19:05:21.148571 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.148545    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 19:05:21.148647 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.148636    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5v5dv\""
Jul 02 19:05:21.149711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.149357    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 19:05:21.149711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.149375    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 19:05:21.149711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.149573    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 19:05:21.149711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.149709    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 19:05:21.165001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.164976    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-72xwt"
Jul 02 19:05:21.173599 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.173556    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-72xwt"
Jul 02 19:05:21.226375 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.226354    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 19:05:21.236881 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.236851    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26ec4b82-e24c-4d87-b690-91add3af40c4-host\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.237031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.236892    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-systemd-units\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.237031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.236918    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysctl-conf\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.237031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.236947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-hostroot\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237003    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-system-cni-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237040    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-etc-selinux\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237105    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-system-cni-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237141    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237168    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-etc-selinux\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237185    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-systemd\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237009    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-hostroot\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237236    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-systemd\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237278    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysconfig\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237339    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-os-release\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237393    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-var-lib-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237420    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-cni-binary-copy\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237434    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-os-release\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237443    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-socket-dir-parent\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237465    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-cni-bin\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237486    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-kubelet\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237507    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-conf-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237539    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ttv2\" (UniqueName: \"kubernetes.io/projected/bd214edd-b8f8-4814-accb-dd631b0f402d-kube-api-access-8ttv2\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237552    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-socket-dir-parent\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237558    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-cni-bin\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-cni-netd\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237593    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-kubelet\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237616    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d31aa88e-4093-4102-915e-ccd7cbd19178-cni-binary-copy\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237631    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-conf-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.237649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-netns\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237659    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26ec4b82-e24c-4d87-b690-91add3af40c4-serviceca\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237710    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-run-ovn-kubernetes\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237736    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237760    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xb64\" (UniqueName: \"kubernetes.io/projected/e757a13e-15b8-4415-bcf3-acc95b235a71-kube-api-access-2xb64\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237772    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-netns\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-modprobe-d\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237841    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-tmp\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237873    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-k8s-cni-cncf-io\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237921    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-k8s-cni-cncf-io\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237933    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-cni-binary-copy\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237949    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-cnibin\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cjxv\" (UniqueName: \"kubernetes.io/projected/e9336630-072c-4de9-af56-044b1f64ecef-kube-api-access-2cjxv\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238015    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-socket-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238038    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovn-node-metrics-cert\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238060    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/575a34d7-8a37-4869-8082-5383d427e35b-dbus\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238082    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-var-lib-kubelet\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.238322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238105    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xp6\" (UniqueName: \"kubernetes.io/projected/d31aa88e-4093-4102-915e-ccd7cbd19178-kube-api-access-d8xp6\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.237992    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-cnibin\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238127    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-os-release\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238175    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d31aa88e-4093-4102-915e-ccd7cbd19178-cni-binary-copy\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238191    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238213    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-kubelet\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238230    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-socket-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238239    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-ovn\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238256    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-os-release\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238303    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-node-log\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238387    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-cni-bin\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238428    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-cnibin\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238481    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-multus-certs\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238485    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238514    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-sys-fs\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238542    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd214edd-b8f8-4814-accb-dd631b0f402d-tmp-dir\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.239070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238553    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-cnibin\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238567    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e0ad0df9-07c9-4a65-87a3-ee31c024ca6a-konnectivity-ca\") pod \"konnectivity-agent-c6czg\" (UID: \"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a\") " pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238572    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-run-multus-certs\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238589    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-slash\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238600    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-sys-fs\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238617    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-run-netns\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238641    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-etc-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238683    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238709    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-kubernetes\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238719    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238724    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-host\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238762    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmnb9\" (UniqueName: \"kubernetes.io/projected/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-kube-api-access-tmnb9\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238784    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-daemon-config\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238799    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-registration-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238833    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/473b186a-061e-4255-9d89-ed1f0e875452-host-slash\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238853    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kvwj\" (UniqueName: \"kubernetes.io/projected/473b186a-061e-4255-9d89-ed1f0e875452-kube-api-access-2kvwj\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238880    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-device-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.239893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238900    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e0ad0df9-07c9-4a65-87a3-ee31c024ca6a-agent-certs\") pod \"konnectivity-agent-c6czg\" (UID: \"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a\") " pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238952    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-log-socket\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238974    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovnkube-config\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.238992    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-sys\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239020    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-lib-modules\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239045    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-cni-multus\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239057    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/473b186a-061e-4255-9d89-ed1f0e875452-host-slash\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239067    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/473b186a-061e-4255-9d89-ed1f0e875452-iptables-alerter-script\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239142    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239285    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-device-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239305    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-registration-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239388    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/575a34d7-8a37-4869-8082-5383d427e35b-kubelet-config\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239417    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysctl-d\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239438    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-tuned\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239461    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-env-overrides\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239492    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovnkube-script-lib\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5ghn\" (UniqueName: \"kubernetes.io/projected/1cc06332-3f9d-4f62-bacb-4c0bd7353000-kube-api-access-g5ghn\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.240452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239538    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-cni-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239592    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22hfb\" (UniqueName: \"kubernetes.io/projected/5fdc6368-8b21-43d3-9809-4c219ad9f06d-kube-api-access-22hfb\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239614    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd214edd-b8f8-4814-accb-dd631b0f402d-hosts-file\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239636    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-582d8\" (UniqueName: \"kubernetes.io/projected/26ec4b82-e24c-4d87-b690-91add3af40c4-kube-api-access-582d8\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239658    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-run\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239675    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-etc-kubernetes\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239714    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-system-cni-dir\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239729    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239786    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5fdc6368-8b21-43d3-9809-4c219ad9f06d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239816    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-etc-kubernetes\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239823    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/473b186a-061e-4255-9d89-ed1f0e875452-iptables-alerter-script\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239847    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-host-var-lib-cni-multus\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.239958    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-cni-dir\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.240248    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9336630-072c-4de9-af56-044b1f64ecef-system-cni-dir\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.240357    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d31aa88e-4093-4102-915e-ccd7cbd19178-multus-daemon-config\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.241109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.240408    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9336630-072c-4de9-af56-044b1f64ecef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.245266 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.245231    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 19:05:21.249378 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.249251    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kvwj\" (UniqueName: \"kubernetes.io/projected/473b186a-061e-4255-9d89-ed1f0e875452-kube-api-access-2kvwj\") pod \"iptables-alerter-2mzmt\" (UID: \"473b186a-061e-4255-9d89-ed1f0e875452\") " pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.249378 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.249348    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8xp6\" (UniqueName: \"kubernetes.io/projected/d31aa88e-4093-4102-915e-ccd7cbd19178-kube-api-access-d8xp6\") pod \"multus-7hw4t\" (UID: \"d31aa88e-4093-4102-915e-ccd7cbd19178\") " pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.249378 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.249351    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cjxv\" (UniqueName: \"kubernetes.io/projected/e9336630-072c-4de9-af56-044b1f64ecef-kube-api-access-2cjxv\") pod \"multus-additional-cni-plugins-dft29\" (UID: \"e9336630-072c-4de9-af56-044b1f64ecef\") " pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.249560 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.249431    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22hfb\" (UniqueName: \"kubernetes.io/projected/5fdc6368-8b21-43d3-9809-4c219ad9f06d-kube-api-access-22hfb\") pod \"aws-ebs-csi-driver-node-fmzs8\" (UID: \"5fdc6368-8b21-43d3-9809-4c219ad9f06d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.340517 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340484    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-env-overrides\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.340517 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340521    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovnkube-script-lib\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.340743 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340540    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5ghn\" (UniqueName: \"kubernetes.io/projected/1cc06332-3f9d-4f62-bacb-4c0bd7353000-kube-api-access-g5ghn\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.340743 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340654    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd214edd-b8f8-4814-accb-dd631b0f402d-hosts-file\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.340743 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340692    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-582d8\" (UniqueName: \"kubernetes.io/projected/26ec4b82-e24c-4d87-b690-91add3af40c4-kube-api-access-582d8\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.340743 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340722    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-run\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340748    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26ec4b82-e24c-4d87-b690-91add3af40c4-host\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340771    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-systemd-units\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340779    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd214edd-b8f8-4814-accb-dd631b0f402d-hosts-file\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340795    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysctl-conf\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340824    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340835    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26ec4b82-e24c-4d87-b690-91add3af40c4-host\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340847    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-systemd\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340871    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-systemd\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340885    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-run\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.340904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340893    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysconfig\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340917    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-var-lib-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340965    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-systemd\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.340969    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttv2\" (UniqueName: \"kubernetes.io/projected/bd214edd-b8f8-4814-accb-dd631b0f402d-kube-api-access-8ttv2\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341026    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-systemd\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341039    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-systemd-units\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341058    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-cni-netd\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341093    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-cni-netd\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341096    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26ec4b82-e24c-4d87-b690-91add3af40c4-serviceca\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341127    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-run-ovn-kubernetes\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.341143    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341170    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341174    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovnkube-script-lib\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341196    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xb64\" (UniqueName: \"kubernetes.io/projected/e757a13e-15b8-4415-bcf3-acc95b235a71-kube-api-access-2xb64\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341219    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-var-lib-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341229    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-run-ovn-kubernetes\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341221    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-modprobe-d\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.341347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341229    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysctl-conf\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.341244    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:21.84122142 +0000 UTC m=+2.991253401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341233    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-env-overrides\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.341288    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341330    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-modprobe-d\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341335    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysconfig\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341338    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-tmp\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.341367    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:05:21.841352673 +0000 UTC m=+2.991384646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341497    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26ec4b82-e24c-4d87-b690-91add3af40c4-serviceca\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341558    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovn-node-metrics-cert\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/575a34d7-8a37-4869-8082-5383d427e35b-dbus\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341590    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-var-lib-kubelet\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-kubelet\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341634    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-ovn\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341657    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-node-log\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341657    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-var-lib-kubelet\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341659    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/575a34d7-8a37-4869-8082-5383d427e35b-dbus\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.342111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341671    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-kubelet\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341681    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-cni-bin\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341698    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-ovn\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341724    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-cni-bin\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341704    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-node-log\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341718    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd214edd-b8f8-4814-accb-dd631b0f402d-tmp-dir\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341761    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e0ad0df9-07c9-4a65-87a3-ee31c024ca6a-konnectivity-ca\") pod \"konnectivity-agent-c6czg\" (UID: \"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a\") " pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341786    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-slash\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341809    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-run-netns\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341834    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-etc-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341858    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341863    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-run-netns\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341835    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-slash\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341889    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-kubernetes\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341893    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-etc-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341911    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-run-openvswitch\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-host\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341940    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmnb9\" (UniqueName: \"kubernetes.io/projected/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-kube-api-access-tmnb9\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.342954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341953    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-kubernetes\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341969    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e0ad0df9-07c9-4a65-87a3-ee31c024ca6a-agent-certs\") pod \"konnectivity-agent-c6czg\" (UID: \"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a\") " pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.341991    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-log-socket\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342000    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd214edd-b8f8-4814-accb-dd631b0f402d-tmp-dir\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovnkube-config\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342037    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-sys\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342074    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-lib-modules\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342104    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-sys\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342104    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342136    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/575a34d7-8a37-4869-8082-5383d427e35b-kubelet-config\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342193    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysctl-d\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342217    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-tuned\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342267    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-lib-modules\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342317    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-log-socket\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342139    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cc06332-3f9d-4f62-bacb-4c0bd7353000-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342317    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e0ad0df9-07c9-4a65-87a3-ee31c024ca6a-konnectivity-ca\") pod \"konnectivity-agent-c6czg\" (UID: \"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a\") " pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342002    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-host\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342386    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/575a34d7-8a37-4869-8082-5383d427e35b-kubelet-config\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.343837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342503    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-sysctl-d\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.344778 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.342649    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovnkube-config\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.344778 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.343909    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-tmp\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.344778 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.343993    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1cc06332-3f9d-4f62-bacb-4c0bd7353000-ovn-node-metrics-cert\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.344778 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.344755    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-etc-tuned\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.345299 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.345274    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e0ad0df9-07c9-4a65-87a3-ee31c024ca6a-agent-certs\") pod \"konnectivity-agent-c6czg\" (UID: \"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a\") " pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.352292 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.352270    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:21.352394 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.352299    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:21.352394 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.352312    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:21.352394 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.352373    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:21.852356711 +0000 UTC m=+3.002388687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:21.353724 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.353662    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-582d8\" (UniqueName: \"kubernetes.io/projected/26ec4b82-e24c-4d87-b690-91add3af40c4-kube-api-access-582d8\") pod \"node-ca-kjg2x\" (UID: \"26ec4b82-e24c-4d87-b690-91add3af40c4\") " pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.353898 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.353867    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xb64\" (UniqueName: \"kubernetes.io/projected/e757a13e-15b8-4415-bcf3-acc95b235a71-kube-api-access-2xb64\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.354433 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.354411    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ttv2\" (UniqueName: \"kubernetes.io/projected/bd214edd-b8f8-4814-accb-dd631b0f402d-kube-api-access-8ttv2\") pod \"node-resolver-xztmn\" (UID: \"bd214edd-b8f8-4814-accb-dd631b0f402d\") " pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.354889 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.354871    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5ghn\" (UniqueName: \"kubernetes.io/projected/1cc06332-3f9d-4f62-bacb-4c0bd7353000-kube-api-access-g5ghn\") pod \"ovnkube-node-sgwpk\" (UID: \"1cc06332-3f9d-4f62-bacb-4c0bd7353000\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.354997 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.354969    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmnb9\" (UniqueName: \"kubernetes.io/projected/8b9078d5-2ca6-4ce0-bba2-6b1933725acc-kube-api-access-tmnb9\") pod \"tuned-v2j5n\" (UID: \"8b9078d5-2ca6-4ce0-bba2-6b1933725acc\") " pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.438228 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.438201    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2mzmt"
Jul 02 19:05:21.445537 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.445513    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8"
Jul 02 19:05:21.456175 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.456137    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:21.467871 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.467845    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7hw4t"
Jul 02 19:05:21.474524 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.474507    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dft29"
Jul 02 19:05:21.482044 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.482026    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2j5n"
Jul 02 19:05:21.490653 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.490637    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kjg2x"
Jul 02 19:05:21.496241 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.496224    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xztmn"
Jul 02 19:05:21.499822 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.499806    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:21.634311 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.634276    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:05:21.845733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.845636    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:21.845733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.845694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:21.845928 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.845783    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:21.845928 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.845868    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:22.845846138 +0000 UTC m=+3.995878102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:21.845928 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.845792    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:21.846073 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.845933    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:05:22.84591437 +0000 UTC m=+3.995946345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:21.946285 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:21.946242    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:21.946462 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.946407    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:21.946462 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.946425    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:21.946462 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.946438    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:21.946619 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:21.946510    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:22.946491223 +0000 UTC m=+4.096523187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:22.177265 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.177215    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:00:21 +0000 UTC" deadline="2028-02-05 15:28:02.116431441 +0000 UTC"
Jul 02 19:05:22.177265 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.177261    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13988h22m39.939174467s"
Jul 02 19:05:22.266875 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.266796    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31aa88e_4093_4102_915e_ccd7cbd19178.slice/crio-1aa5d2e3fe2bfcd18cf7b76b5053cd2fbebcbe78fc62141b14cfdc0174b9eaf8 WatchSource:0}: Error finding container 1aa5d2e3fe2bfcd18cf7b76b5053cd2fbebcbe78fc62141b14cfdc0174b9eaf8: Status 404 returned error can't find the container with id 1aa5d2e3fe2bfcd18cf7b76b5053cd2fbebcbe78fc62141b14cfdc0174b9eaf8
Jul 02 19:05:22.270230 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.270202    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ad0df9_07c9_4a65_87a3_ee31c024ca6a.slice/crio-ac9f703f46dc5736efdc572879e83d220ec0db5a547c857f03db6e96dbacb23a WatchSource:0}: Error finding container ac9f703f46dc5736efdc572879e83d220ec0db5a547c857f03db6e96dbacb23a: Status 404 returned error can't find the container with id ac9f703f46dc5736efdc572879e83d220ec0db5a547c857f03db6e96dbacb23a
Jul 02 19:05:22.274260 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.274235    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd214edd_b8f8_4814_accb_dd631b0f402d.slice/crio-b243ae2e45b1dd190d9b80fa818dabd456372fc4e2155687e5932de17915a8b5 WatchSource:0}: Error finding container b243ae2e45b1dd190d9b80fa818dabd456372fc4e2155687e5932de17915a8b5: Status 404 returned error can't find the container with id b243ae2e45b1dd190d9b80fa818dabd456372fc4e2155687e5932de17915a8b5
Jul 02 19:05:22.274659 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.274635    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473b186a_061e_4255_9d89_ed1f0e875452.slice/crio-7e12d81d8dc46ec1ab913858cbaf6d0c3f884bb11116432ce71feb0bf31924a8 WatchSource:0}: Error finding container 7e12d81d8dc46ec1ab913858cbaf6d0c3f884bb11116432ce71feb0bf31924a8: Status 404 returned error can't find the container with id 7e12d81d8dc46ec1ab913858cbaf6d0c3f884bb11116432ce71feb0bf31924a8
Jul 02 19:05:22.275738 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.275212    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9336630_072c_4de9_af56_044b1f64ecef.slice/crio-d53f9825c76f7c8aeeeb74dc759886bd389e705b5afdaa8d2f045214182ca6d8 WatchSource:0}: Error finding container d53f9825c76f7c8aeeeb74dc759886bd389e705b5afdaa8d2f045214182ca6d8: Status 404 returned error can't find the container with id d53f9825c76f7c8aeeeb74dc759886bd389e705b5afdaa8d2f045214182ca6d8
Jul 02 19:05:22.276223 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.276198    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc06332_3f9d_4f62_bacb_4c0bd7353000.slice/crio-fea1a305c5c51714b483e529e848ba0767b1f54faf92c0c190f4d3b6affa5a01 WatchSource:0}: Error finding container fea1a305c5c51714b483e529e848ba0767b1f54faf92c0c190f4d3b6affa5a01: Status 404 returned error can't find the container with id fea1a305c5c51714b483e529e848ba0767b1f54faf92c0c190f4d3b6affa5a01
Jul 02 19:05:22.277414 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.277384    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdc6368_8b21_43d3_9809_4c219ad9f06d.slice/crio-1d74e879568b9cb9e5739962f1a9008499c5923597f5a3fe2a4a344948245ea2 WatchSource:0}: Error finding container 1d74e879568b9cb9e5739962f1a9008499c5923597f5a3fe2a4a344948245ea2: Status 404 returned error can't find the container with id 1d74e879568b9cb9e5739962f1a9008499c5923597f5a3fe2a4a344948245ea2
Jul 02 19:05:22.278365 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.278057    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ec4b82_e24c_4d87_b690_91add3af40c4.slice/crio-ca7ac5ed59d0a024bbcbc892d256c66f82869121e554e7adb414a1b48d2fa1aa WatchSource:0}: Error finding container ca7ac5ed59d0a024bbcbc892d256c66f82869121e554e7adb414a1b48d2fa1aa: Status 404 returned error can't find the container with id ca7ac5ed59d0a024bbcbc892d256c66f82869121e554e7adb414a1b48d2fa1aa
Jul 02 19:05:22.279508 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:22.279480    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9078d5_2ca6_4ce0_bba2_6b1933725acc.slice/crio-f21ecfb934975a3d4478b444ff3179324e6dee241b31c89b4cf93e4ebffc458b WatchSource:0}: Error finding container f21ecfb934975a3d4478b444ff3179324e6dee241b31c89b4cf93e4ebffc458b: Status 404 returned error can't find the container with id f21ecfb934975a3d4478b444ff3179324e6dee241b31c89b4cf93e4ebffc458b
Jul 02 19:05:22.309493 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.309468    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:22.309615 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.309593    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:22.316724 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.316702    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2j5n" event={"ID":"8b9078d5-2ca6-4ce0-bba2-6b1933725acc","Type":"ContainerStarted","Data":"f21ecfb934975a3d4478b444ff3179324e6dee241b31c89b4cf93e4ebffc458b"}
Jul 02 19:05:22.317782 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.317764    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kjg2x" event={"ID":"26ec4b82-e24c-4d87-b690-91add3af40c4","Type":"ContainerStarted","Data":"ca7ac5ed59d0a024bbcbc892d256c66f82869121e554e7adb414a1b48d2fa1aa"}
Jul 02 19:05:22.318872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.318853    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8" event={"ID":"5fdc6368-8b21-43d3-9809-4c219ad9f06d","Type":"ContainerStarted","Data":"1d74e879568b9cb9e5739962f1a9008499c5923597f5a3fe2a4a344948245ea2"}
Jul 02 19:05:22.324200 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.324180    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"fea1a305c5c51714b483e529e848ba0767b1f54faf92c0c190f4d3b6affa5a01"}
Jul 02 19:05:22.326085 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.325919    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2mzmt" event={"ID":"473b186a-061e-4255-9d89-ed1f0e875452","Type":"ContainerStarted","Data":"7e12d81d8dc46ec1ab913858cbaf6d0c3f884bb11116432ce71feb0bf31924a8"}
Jul 02 19:05:22.327014 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.326986    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xztmn" event={"ID":"bd214edd-b8f8-4814-accb-dd631b0f402d","Type":"ContainerStarted","Data":"b243ae2e45b1dd190d9b80fa818dabd456372fc4e2155687e5932de17915a8b5"}
Jul 02 19:05:22.327969 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.327949    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c6czg" event={"ID":"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a","Type":"ContainerStarted","Data":"ac9f703f46dc5736efdc572879e83d220ec0db5a547c857f03db6e96dbacb23a"}
Jul 02 19:05:22.328868 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.328844    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerStarted","Data":"d53f9825c76f7c8aeeeb74dc759886bd389e705b5afdaa8d2f045214182ca6d8"}
Jul 02 19:05:22.329826 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.329808    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7hw4t" event={"ID":"d31aa88e-4093-4102-915e-ccd7cbd19178","Type":"ContainerStarted","Data":"1aa5d2e3fe2bfcd18cf7b76b5053cd2fbebcbe78fc62141b14cfdc0174b9eaf8"}
Jul 02 19:05:22.852579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.852495    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:22.852579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.852552    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:22.852814 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.852724    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:22.852814 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.852789    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:05:24.852768295 +0000 UTC m=+6.002800259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:22.853232 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.853210    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:22.853327 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.853269    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:24.853252638 +0000 UTC m=+6.003284603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:22.953361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:22.953325    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:22.953548 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.953528    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:22.953612 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.953557    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:22.953612 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.953571    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:22.953707 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:22.953632    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:24.953612796 +0000 UTC m=+6.103644772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:23.178067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.178026    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:00:21 +0000 UTC" deadline="2028-04-12 20:20:16.470876008 +0000 UTC"
Jul 02 19:05:23.178067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.178066    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15601h14m53.292814329s"
Jul 02 19:05:23.310298 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.310266    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:23.310475 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:23.310407    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:23.310913 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.310894    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:23.311027 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:23.311008    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:23.358840 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.358806    2585 generic.go:356] "Generic (PLEG): container finished" podID="91bf6946301bf34241dbf61205b71188" containerID="a3972921b245e1416f0542d978eee74d9141d24eefb2bb86748227a92002b819" exitCode=0
Jul 02 19:05:23.359013 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.358909    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"91bf6946301bf34241dbf61205b71188","Type":"ContainerDied","Data":"a3972921b245e1416f0542d978eee74d9141d24eefb2bb86748227a92002b819"}
Jul 02 19:05:23.377642 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.377597    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" event={"ID":"4ad4923ba204444f53390e622f229eba","Type":"ContainerStarted","Data":"7ffbdd2dd5ea39f40448cb53efff6b856f2a4fe6a60da3dbf8f675e4cc78484b"}
Jul 02 19:05:23.393521 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:23.393472    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" podStartSLOduration=3.393457087 podStartE2EDuration="3.393457087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:23.393326487 +0000 UTC m=+4.543358496" watchObservedRunningTime="2026-07-02 19:05:23.393457087 +0000 UTC m=+4.543489099"
Jul 02 19:05:24.309456 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:24.309416    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:24.309915 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.309554    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:24.397460 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:24.397312    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"91bf6946301bf34241dbf61205b71188","Type":"ContainerStarted","Data":"4885a77ab57e6bfb620fc50ce413cd17dc6a36b902c5bd41ef4db1bddd3e9635"}
Jul 02 19:05:24.871966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:24.871924    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:24.872177 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:24.871975    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:24.872177 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.872073    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:24.872177 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.872135    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:24.872177 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.872159    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:28.872123315 +0000 UTC m=+10.022155291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:24.872391 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.872219    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:05:28.872202356 +0000 UTC m=+10.022234317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:24.973320 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:24.973271    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:24.973505 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.973452    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:24.973505 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.973472    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:24.973505 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.973485    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:24.973667 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:24.973544    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:28.973525602 +0000 UTC m=+10.123557565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:25.310753 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:25.310065    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:25.310753 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:25.310216    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:25.310753 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:25.310615    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:25.310753 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:25.310706    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:26.309718 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:26.309221    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:26.309718 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:26.309358    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:27.309702 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:27.309603    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:27.310122 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:27.309742    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:27.310122 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:27.309615    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:27.310252 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:27.310194    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:28.309341 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:28.309308    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:28.309523 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:28.309436    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:28.907273 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:28.906675    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:28.907273 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:28.906727    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:28.907273 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:28.906868    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:28.907273 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:28.906931    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:05:36.906912022 +0000 UTC m=+18.056943983 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:28.907273 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:28.907091    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:28.907273 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:28.907183    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:36.907164244 +0000 UTC m=+18.057196220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:29.007641 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:29.007559    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:29.011174 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:29.008285    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:29.011174 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:29.008319    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:29.011174 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:29.008345    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:29.011174 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:29.008417    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:37.008390027 +0000 UTC m=+18.158422011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:29.310816 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:29.310236    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:29.310816 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:29.310359    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:29.311381 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:29.311143    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:29.311381 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:29.311303    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:30.309529 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:30.309499    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:30.309973 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:30.309628    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:31.310004 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:31.309973    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:31.310490 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:31.310104    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:31.310551 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:31.310487    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:31.310651 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:31.310595    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:32.309542 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:32.309509    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:32.309709 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:32.309640    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:33.309748 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:33.309707    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:33.310220 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:33.309858    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:33.310220 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:33.309901    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:33.310220 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:33.310038    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:34.309119 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:34.309077    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:34.309309 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:34.309229    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:35.309244 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:35.309208    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:35.309692 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:35.309208    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:35.309692 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:35.309330    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:35.309692 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:35.309407    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:36.309680 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:36.309642    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:36.310183 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:36.309761    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:36.969762 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:36.969715    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:36.969762 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:36.969765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:36.970007 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:36.969873    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:36.970007 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:36.969929    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:05:52.969914196 +0000 UTC m=+34.119946157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:36.970007 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:36.969873    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:36.970255 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:36.970019    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:52.970002306 +0000 UTC m=+34.120034269 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:37.071020 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:37.070982    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:37.071225 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:37.071180    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:37.071225 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:37.071200    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:37.071225 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:37.071209    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:37.071377 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:37.071261    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:53.071245738 +0000 UTC m=+34.221277701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:37.309178 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:37.309082    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:37.309320 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:37.309241    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:37.309320 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:37.309259    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:37.309440 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:37.309345    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:38.309573 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:38.309538    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:38.310050 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:38.309678    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:39.310719 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:39.310680    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:39.311142 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:39.310814    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:39.311142 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:39.310853    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:39.311142 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:39.310928    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:40.309625 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:40.309585    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:40.309827 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:40.309718    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:41.309326 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.309011    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:41.309995 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.309010    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:41.309995 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:41.309432    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:41.309995 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:41.309481    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:41.428741 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.428715    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xztmn" event={"ID":"bd214edd-b8f8-4814-accb-dd631b0f402d","Type":"ContainerStarted","Data":"f84876c49bc03eef57a3cc0897627cade31424d4132de2373fc02628dcdacd7d"}
Jul 02 19:05:41.430022 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.429994    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c6czg" event={"ID":"e0ad0df9-07c9-4a65-87a3-ee31c024ca6a","Type":"ContainerStarted","Data":"21ae30f09812f8a8f06fbc134d95b4bbc660bd56b9d76922d0845febd9f57151"}
Jul 02 19:05:41.431361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.431338    2585 generic.go:356] "Generic (PLEG): container finished" podID="e9336630-072c-4de9-af56-044b1f64ecef" containerID="57a81d63ad5c7d486cac845f3fce2ccfafadbc517913cd55a2d71f591352fbd6" exitCode=0
Jul 02 19:05:41.431448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.431413    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerDied","Data":"57a81d63ad5c7d486cac845f3fce2ccfafadbc517913cd55a2d71f591352fbd6"}
Jul 02 19:05:41.432836 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.432818    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7hw4t" event={"ID":"d31aa88e-4093-4102-915e-ccd7cbd19178","Type":"ContainerStarted","Data":"55c985e9f95330c0721a7001603027954ed29fa3cc7c093a4e7eacdac8846f87"}
Jul 02 19:05:41.434337 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.434313    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2j5n" event={"ID":"8b9078d5-2ca6-4ce0-bba2-6b1933725acc","Type":"ContainerStarted","Data":"9949dd8c99b526a3d8382cae1df726b9631f16edcb785b812510871f8e126640"}
Jul 02 19:05:41.435753 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.435729    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kjg2x" event={"ID":"26ec4b82-e24c-4d87-b690-91add3af40c4","Type":"ContainerStarted","Data":"4ff251e0cf1227a087b424d8230a75cb64bdfd3ca94fdeb9d9ec556e02980c42"}
Jul 02 19:05:41.437177 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.437139    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8" event={"ID":"5fdc6368-8b21-43d3-9809-4c219ad9f06d","Type":"ContainerStarted","Data":"e871ead6e5fb3f14919069036b2f1b3f32baff1411c699d38800a1e2b873e59d"}
Jul 02 19:05:41.440177 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.440137    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"f43d9634091a543bcab3794712b71a22a330f038c4437bec9934a1b1db7b7b2e"}
Jul 02 19:05:41.440241 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.440210    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"16c36dbc219dc27f47dc2329ebe8124878345ee3ae81d0640a76d7aac135adaa"}
Jul 02 19:05:41.440274 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.440257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"b82442a58370746eccd825b1397301bc3f5398aa578aa9bf7460f5c1d9960745"}
Jul 02 19:05:41.440303 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.440290    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"d924cb11b2d49bab13e8d07637062e6ebf060ed1ae979c6f267718f6a0591d3f"}
Jul 02 19:05:41.443579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.443537    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" podStartSLOduration=21.443522129 podStartE2EDuration="21.443522129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:24.412981527 +0000 UTC m=+5.563013513" watchObservedRunningTime="2026-07-02 19:05:41.443522129 +0000 UTC m=+22.593554113"
Jul 02 19:05:41.444180 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.444125    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xztmn" podStartSLOduration=3.842302729 podStartE2EDuration="22.44411528s" totalImagesPullingTime="18.601812551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:41.443459266 +0000 UTC m=+22.593491249" watchObservedRunningTime="2026-07-02 19:05:41.44411528 +0000 UTC m=+22.594147265"
Jul 02 19:05:41.461872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.461835    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v2j5n" podStartSLOduration=3.865249424 podStartE2EDuration="22.461819065s" totalImagesPullingTime="18.596569641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:41.461215917 +0000 UTC m=+22.611247900" watchObservedRunningTime="2026-07-02 19:05:41.461819065 +0000 UTC m=+22.611851048"
Jul 02 19:05:41.477808 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.477767    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kjg2x" podStartSLOduration=3.8804591999999998 podStartE2EDuration="22.477758791s" totalImagesPullingTime="18.597299591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:41.477556167 +0000 UTC m=+22.627588150" watchObservedRunningTime="2026-07-02 19:05:41.477758791 +0000 UTC m=+22.627790775"
Jul 02 19:05:41.497541 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.497503    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7hw4t" podStartSLOduration=3.838959248 podStartE2EDuration="22.497488756s" totalImagesPullingTime="18.658529508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:41.497158599 +0000 UTC m=+22.647190574" watchObservedRunningTime="2026-07-02 19:05:41.497488756 +0000 UTC m=+22.647520738"
Jul 02 19:05:41.532813 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:41.532762    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c6czg" podStartSLOduration=8.418017383 podStartE2EDuration="22.532741487s" totalImagesPullingTime="14.114724104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:41.532310638 +0000 UTC m=+22.682342622" watchObservedRunningTime="2026-07-02 19:05:41.532741487 +0000 UTC m=+22.682773473"
Jul 02 19:05:42.309315 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.309286    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:42.309494 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:42.309407    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:42.424629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.424589    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 19:05:42.444904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.444868    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8" event={"ID":"5fdc6368-8b21-43d3-9809-4c219ad9f06d","Type":"ContainerStarted","Data":"d628a2f03f46a4f15406d45867031dbf27e7f03d0d620bfaa005973b537db80d"}
Jul 02 19:05:42.448240 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.448210    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"af781d5844f2d6f215e715b27ce7a90d5b59e3c108cf6e5d9025c976245c9f58"}
Jul 02 19:05:42.448354 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.448285    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"7e40c20271c47c433bbf3f9b817506f116052e0848c1c4437ebddd7389344d06"}
Jul 02 19:05:42.449807 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.449736    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2mzmt" event={"ID":"473b186a-061e-4255-9d89-ed1f0e875452","Type":"ContainerStarted","Data":"3ab8b753432b4ce7981075180c569e6a8c8343658861a21499ed11c449c3a240"}
Jul 02 19:05:42.464817 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:42.464769    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2mzmt" podStartSLOduration=4.8640702860000005 podStartE2EDuration="23.464753213s" totalImagesPullingTime="18.600682927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:42.46449472 +0000 UTC m=+23.614526701" watchObservedRunningTime="2026-07-02 19:05:42.464753213 +0000 UTC m=+23.614785197"
Jul 02 19:05:43.194492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.194376    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:05:42.424615554Z","UUID":"260a8053-3c8b-4b48-a081-1b42e2cf2ba2","Handler":null,"Name":"","Endpoint":""}
Jul 02 19:05:43.196188 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.196165    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 19:05:43.196312 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.196206    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 19:05:43.218441 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.218402    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:43.219035 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.219006    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:43.309000 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.308973    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:43.309204 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.309004    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:43.309204 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:43.309078    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:43.309204 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:43.309164    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:43.452127 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.452053    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:43.452679 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:43.452405    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c6czg"
Jul 02 19:05:44.309812 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:44.309778    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:44.310059 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:44.309884    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:44.455604 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:44.455561    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8" event={"ID":"5fdc6368-8b21-43d3-9809-4c219ad9f06d","Type":"ContainerStarted","Data":"56e79bb82f2da9b2933d81fb3068e9e0f1bce8228c45bde0d6300783ef1d08ac"}
Jul 02 19:05:44.458677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:44.458654    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"d48ead72e5120aed3a5ab61a4fdb04417c1564adb0e3ca5d28c2544a305aa2e4"}
Jul 02 19:05:45.309182 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:45.308957    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:45.309367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:45.309017    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:45.309367 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:45.309286    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:45.309367 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:45.309335    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:46.309799 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:46.309761    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:46.310187 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:46.309894    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:47.309651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.309444    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:47.309855 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.309497    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:47.309855 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:47.309758    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:47.309855 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:47.309806    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:47.468576 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.468539    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" event={"ID":"1cc06332-3f9d-4f62-bacb-4c0bd7353000","Type":"ContainerStarted","Data":"f6ae768172191bd40a227fa940f5dec07395687916196a939307e69cec205dd9"}
Jul 02 19:05:47.468783 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.468766    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:47.473459 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.473428    2585 generic.go:356] "Generic (PLEG): container finished" podID="e9336630-072c-4de9-af56-044b1f64ecef" containerID="1c162e2c307122ee22b98c734157cd19aeb75f171aef262150781675a093e185" exitCode=0
Jul 02 19:05:47.473577 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.473474    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerDied","Data":"1c162e2c307122ee22b98c734157cd19aeb75f171aef262150781675a093e185"}
Jul 02 19:05:47.485349 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.485320    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:47.498680 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.498635    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk" podStartSLOduration=9.82717329 podStartE2EDuration="28.498623385s" totalImagesPullingTime="18.671450095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:47.498535206 +0000 UTC m=+28.648567189" watchObservedRunningTime="2026-07-02 19:05:47.498623385 +0000 UTC m=+28.648655399"
Jul 02 19:05:47.498835 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:47.498812    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmzs8" podStartSLOduration=7.665127592 podStartE2EDuration="28.498806393s" totalImagesPullingTime="20.833678801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:44.472538072 +0000 UTC m=+25.622570056" watchObservedRunningTime="2026-07-02 19:05:47.498806393 +0000 UTC m=+28.648838416"
Jul 02 19:05:48.308948 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.308912    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:48.309246 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:48.309043    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:48.476645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.476518    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:48.476645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.476625    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:48.491371 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.491345    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:05:48.834758 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.834664    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fc92r"]
Jul 02 19:05:48.834901 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.834812    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:48.834959 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:48.834917    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:48.835198 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.835176    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vrgc9"]
Jul 02 19:05:48.835263 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.835251    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:48.835332 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:48.835317    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:48.840678 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.837861    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-crgh4"]
Jul 02 19:05:48.840678 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:48.838013    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:48.840678 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:48.838246    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:49.480092 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:49.480057    2585 generic.go:356] "Generic (PLEG): container finished" podID="e9336630-072c-4de9-af56-044b1f64ecef" containerID="e0e7f4af8369491e6ad2dceef511e469f13b212000f8e6f3b3846752d962b2a4" exitCode=0
Jul 02 19:05:49.480653 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:49.480140    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerDied","Data":"e0e7f4af8369491e6ad2dceef511e469f13b212000f8e6f3b3846752d962b2a4"}
Jul 02 19:05:50.309753 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:50.309679    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:50.309893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:50.309680    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:50.309893 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:50.309826    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:50.310004 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:50.309887    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:50.310004 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:50.309681    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:50.310004 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:50.309976    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:51.487532 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:51.487258    2585 generic.go:356] "Generic (PLEG): container finished" podID="e9336630-072c-4de9-af56-044b1f64ecef" containerID="dcfd7f03284b3abbae2f4617c3ff22ab0ea5ac9a290361f38d925ed60a45c14a" exitCode=0
Jul 02 19:05:51.487532 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:51.487345    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerDied","Data":"dcfd7f03284b3abbae2f4617c3ff22ab0ea5ac9a290361f38d925ed60a45c14a"}
Jul 02 19:05:52.309880 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:52.309842    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:52.309880 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:52.309871    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:52.310122 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:52.309862    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:52.310122 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.309976    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrgc9" podUID="10c62fd6-7471-43e6-9036-f7351746d419"
Jul 02 19:05:52.310122 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.310051    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fc92r" podUID="575a34d7-8a37-4869-8082-5383d427e35b"
Jul 02 19:05:52.310269 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.310161    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-crgh4" podUID="e757a13e-15b8-4415-bcf3-acc95b235a71"
Jul 02 19:05:52.996333 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:52.996299    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:52.996782 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:52.996353    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:52.996782 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.996462    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:52.996782 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.996468    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:52.996782 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.996517    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret podName:575a34d7-8a37-4869-8082-5383d427e35b nodeName:}" failed. No retries permitted until 2026-07-02 19:06:24.996504332 +0000 UTC m=+66.146536293 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret") pod "global-pull-secret-syncer-fc92r" (UID: "575a34d7-8a37-4869-8082-5383d427e35b") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:05:52.996782 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:52.996538    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs podName:e757a13e-15b8-4415-bcf3-acc95b235a71 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:24.996527597 +0000 UTC m=+66.146559558 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs") pod "network-metrics-daemon-crgh4" (UID: "e757a13e-15b8-4415-bcf3-acc95b235a71") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:05:53.097176 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.097128    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:53.097349 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.097331    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:05:53.097407 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.097357    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:05:53.097407 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.097371    2585 projected.go:196] Error preparing data for projected volume kube-api-access-gx8tr for pod openshift-network-diagnostics/network-check-target-vrgc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:53.097500 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.097434    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr podName:10c62fd6-7471-43e6-9036-f7351746d419 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:25.097416525 +0000 UTC m=+66.247448509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx8tr" (UniqueName: "kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr") pod "network-check-target-vrgc9" (UID: "10c62fd6-7471-43e6-9036-f7351746d419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:05:53.663425 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.663400    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeReady"
Jul 02 19:05:53.663589 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.663554    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 19:05:53.713101 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.713064    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64447cc5fc-65lzg"]
Jul 02 19:05:53.716424 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.716402    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dmlds"]
Jul 02 19:05:53.716555 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.716533    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.719475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.719454    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 02 19:05:53.719596 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.719481    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-68mj4\""
Jul 02 19:05:53.719655 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.719641    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 02 19:05:53.719712 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.719685    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 02 19:05:53.720301 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.720284    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jwx7v"]
Jul 02 19:05:53.720426 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.720404    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.726794 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.725199    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 19:05:53.726794 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.726445    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crw8l\""
Jul 02 19:05:53.728269 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.727923    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:53.728269 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.728123    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 19:05:53.728540 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.728523    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 02 19:05:53.729252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.729205    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jwx7v"]
Jul 02 19:05:53.730102 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.730082    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dmlds"]
Jul 02 19:05:53.734504 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.734481    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 19:05:53.734504 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.734481    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 19:05:53.734675 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.734593    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 19:05:53.734733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.734720    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h97jw\""
Jul 02 19:05:53.736431 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.736410    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64447cc5fc-65lzg"]
Jul 02 19:05:53.802488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802454    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802490    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hjgb\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-kube-api-access-5hjgb\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802512    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-config-volume\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-image-registry-private-configuration\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802576    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-certificates\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802601    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-installation-pull-secrets\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802619    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-tmp-dir\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d9e565-fc54-4f19-b4f4-bf30cb043466-ca-trust-extracted\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802659    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sx4k\" (UniqueName: \"kubernetes.io/projected/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-kube-api-access-4sx4k\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.802872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802708    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.802872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802747    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-trusted-ca\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.802872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.802771    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-bound-sa-token\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.903915 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.903880    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.903926    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hjgb\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-kube-api-access-5hjgb\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.903951    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-config-volume\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.903976    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36604680-9a15-4463-aaa5-721782988b0a-cert\") pod \"ingress-canary-jwx7v\" (UID: \"36604680-9a15-4463-aaa5-721782988b0a\") " pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904002    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-image-registry-private-configuration\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904029    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-certificates\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.904032    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-installation-pull-secrets\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904076 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.904059    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64447cc5fc-65lzg: secret "image-registry-tls" not found
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904107    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-tmp-dir\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.904121    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls podName:a3d9e565-fc54-4f19-b4f4-bf30cb043466 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:54.404099549 +0000 UTC m=+35.554131512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls") pod "image-registry-64447cc5fc-65lzg" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466") : secret "image-registry-tls" not found
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904184    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d9e565-fc54-4f19-b4f4-bf30cb043466-ca-trust-extracted\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904215    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sx4k\" (UniqueName: \"kubernetes.io/projected/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-kube-api-access-4sx4k\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904254    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55msl\" (UniqueName: \"kubernetes.io/projected/36604680-9a15-4463-aaa5-721782988b0a-kube-api-access-55msl\") pod \"ingress-canary-jwx7v\" (UID: \"36604680-9a15-4463-aaa5-721782988b0a\") " pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904290    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904334    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-trusted-ca\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904367    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-bound-sa-token\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904565 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.904570    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:05:53.904957 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:53.904617    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls podName:99f84018-e5e1-4d3e-8b4d-7c96d34480d3 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:54.404599258 +0000 UTC m=+35.554631219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls") pod "dns-default-dmlds" (UID: "99f84018-e5e1-4d3e-8b4d-7c96d34480d3") : secret "dns-default-metrics-tls" not found
Jul 02 19:05:53.904957 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904669    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d9e565-fc54-4f19-b4f4-bf30cb043466-ca-trust-extracted\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.904957 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.904860    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-config-volume\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.905075 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.905044    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-tmp-dir\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.905140 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.905116    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-certificates\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.905499 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.905476    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-trusted-ca\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.908873 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.908854    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-installation-pull-secrets\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.909230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.909211    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-image-registry-private-configuration\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.912786 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.912757    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sx4k\" (UniqueName: \"kubernetes.io/projected/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-kube-api-access-4sx4k\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:53.913079 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.913057    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-bound-sa-token\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:53.913201 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:53.913163    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hjgb\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-kube-api-access-5hjgb\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:54.005589 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.005510    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36604680-9a15-4463-aaa5-721782988b0a-cert\") pod \"ingress-canary-jwx7v\" (UID: \"36604680-9a15-4463-aaa5-721782988b0a\") " pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:54.005589 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.005556    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55msl\" (UniqueName: \"kubernetes.io/projected/36604680-9a15-4463-aaa5-721782988b0a-kube-api-access-55msl\") pod \"ingress-canary-jwx7v\" (UID: \"36604680-9a15-4463-aaa5-721782988b0a\") " pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:54.008238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.008216    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36604680-9a15-4463-aaa5-721782988b0a-cert\") pod \"ingress-canary-jwx7v\" (UID: \"36604680-9a15-4463-aaa5-721782988b0a\") " pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:54.016012 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.015987    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55msl\" (UniqueName: \"kubernetes.io/projected/36604680-9a15-4463-aaa5-721782988b0a-kube-api-access-55msl\") pod \"ingress-canary-jwx7v\" (UID: \"36604680-9a15-4463-aaa5-721782988b0a\") " pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:54.047045 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.047018    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jwx7v"
Jul 02 19:05:54.204983 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.204884    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jwx7v"]
Jul 02 19:05:54.309566 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.309311    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:05:54.309715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.309311    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:05:54.309715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.309311    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:05:54.312892 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.312867    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjgvz\""
Jul 02 19:05:54.313011 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.312894    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 19:05:54.313011 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.312982    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mxlv5\""
Jul 02 19:05:54.313011 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.312997    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 19:05:54.313325 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.313302    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 19:05:54.313435 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.313345    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 19:05:54.407728 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.407684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:54.407892 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:54.407785    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:54.407892 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:54.407834    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:05:54.407892 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:54.407883    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:05:54.408049 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:54.407898    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64447cc5fc-65lzg: secret "image-registry-tls" not found
Jul 02 19:05:54.408049 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:54.407906    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls podName:99f84018-e5e1-4d3e-8b4d-7c96d34480d3 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:55.407888489 +0000 UTC m=+36.557920452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls") pod "dns-default-dmlds" (UID: "99f84018-e5e1-4d3e-8b4d-7c96d34480d3") : secret "dns-default-metrics-tls" not found
Jul 02 19:05:54.408049 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:54.407953    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls podName:a3d9e565-fc54-4f19-b4f4-bf30cb043466 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:55.407939842 +0000 UTC m=+36.557971816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls") pod "image-registry-64447cc5fc-65lzg" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466") : secret "image-registry-tls" not found
Jul 02 19:05:55.414521 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:55.414485    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:55.415215 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:55.414554    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:55.415215 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:55.414647    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:05:55.415215 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:55.414655    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:05:55.415215 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:55.414665    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64447cc5fc-65lzg: secret "image-registry-tls" not found
Jul 02 19:05:55.415215 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:55.414721    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls podName:a3d9e565-fc54-4f19-b4f4-bf30cb043466 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:57.414702987 +0000 UTC m=+38.564734955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls") pod "image-registry-64447cc5fc-65lzg" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466") : secret "image-registry-tls" not found
Jul 02 19:05:55.415215 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:55.414744    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls podName:99f84018-e5e1-4d3e-8b4d-7c96d34480d3 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:57.414733791 +0000 UTC m=+38.564765769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls") pod "dns-default-dmlds" (UID: "99f84018-e5e1-4d3e-8b4d-7c96d34480d3") : secret "dns-default-metrics-tls" not found
Jul 02 19:05:55.670281 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:05:55.670208    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36604680_9a15_4463_aaa5_721782988b0a.slice/crio-0ad6ce8039b656738c97586ea064a80dc15ed27e12fa1377cf49d1ea50c7ccd2 WatchSource:0}: Error finding container 0ad6ce8039b656738c97586ea064a80dc15ed27e12fa1377cf49d1ea50c7ccd2: Status 404 returned error can't find the container with id 0ad6ce8039b656738c97586ea064a80dc15ed27e12fa1377cf49d1ea50c7ccd2
Jul 02 19:05:56.502538 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:56.502501    2585 generic.go:356] "Generic (PLEG): container finished" podID="e9336630-072c-4de9-af56-044b1f64ecef" containerID="ca3d2b40a5b86aaa7fe71417d53593c2750497e14dfa4c4208fba81832bea9a7" exitCode=0
Jul 02 19:05:56.502961 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:56.502582    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerDied","Data":"ca3d2b40a5b86aaa7fe71417d53593c2750497e14dfa4c4208fba81832bea9a7"}
Jul 02 19:05:56.503919 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:56.503889    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jwx7v" event={"ID":"36604680-9a15-4463-aaa5-721782988b0a","Type":"ContainerStarted","Data":"0ad6ce8039b656738c97586ea064a80dc15ed27e12fa1377cf49d1ea50c7ccd2"}
Jul 02 19:05:57.430105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:57.430063    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:05:57.430315 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:57.430258    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:05:57.430315 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:57.430279    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64447cc5fc-65lzg: secret "image-registry-tls" not found
Jul 02 19:05:57.430315 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:57.430284    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:05:57.430470 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:57.430344    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls podName:a3d9e565-fc54-4f19-b4f4-bf30cb043466 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:01.43032315 +0000 UTC m=+42.580355128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls") pod "image-registry-64447cc5fc-65lzg" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466") : secret "image-registry-tls" not found
Jul 02 19:05:57.430470 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:57.430388    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:05:57.430470 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:05:57.430439    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls podName:99f84018-e5e1-4d3e-8b4d-7c96d34480d3 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:01.430424356 +0000 UTC m=+42.580456338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls") pod "dns-default-dmlds" (UID: "99f84018-e5e1-4d3e-8b4d-7c96d34480d3") : secret "dns-default-metrics-tls" not found
Jul 02 19:05:57.510638 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:57.510608    2585 generic.go:356] "Generic (PLEG): container finished" podID="e9336630-072c-4de9-af56-044b1f64ecef" containerID="8692c8df6ed8c3e5b16aafddd8f7b8a4ba845e1711a8551682519a1a38aaf153" exitCode=0
Jul 02 19:05:57.510638 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:57.510645    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerDied","Data":"8692c8df6ed8c3e5b16aafddd8f7b8a4ba845e1711a8551682519a1a38aaf153"}
Jul 02 19:05:58.516638 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:58.516555    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dft29" event={"ID":"e9336630-072c-4de9-af56-044b1f64ecef","Type":"ContainerStarted","Data":"c8ff2144f21df251695e16041dba82bdcf23be3df94eedbfeb7ca1e7eb1fd22b"}
Jul 02 19:05:58.517995 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:58.517969    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jwx7v" event={"ID":"36604680-9a15-4463-aaa5-721782988b0a","Type":"ContainerStarted","Data":"7326e91e8df824f4ef89e0cda36d5c5d3817c91a67e73847486a0e2e2c03b3f1"}
Jul 02 19:05:58.558269 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:58.558209    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dft29" podStartSLOduration=9.17414096 podStartE2EDuration="39.558188986s" totalImagesPullingTime="30.384048026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:58.541323676 +0000 UTC m=+39.691355660" watchObservedRunningTime="2026-07-02 19:05:58.558188986 +0000 UTC m=+39.708220969"
Jul 02 19:05:58.558457 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:05:58.558433    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jwx7v" podStartSLOduration=2.975101387 podStartE2EDuration="5.558425596s" totalImagesPullingTime="2.583324209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:58.558119654 +0000 UTC m=+39.708151639" watchObservedRunningTime="2026-07-02 19:05:58.558425596 +0000 UTC m=+39.708457581"
Jul 02 19:06:01.461608 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:01.461562    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:01.462063 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:01.461640    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:01.462063 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:01.461730    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:06:01.462063 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:01.461741    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:06:01.462063 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:01.461753    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64447cc5fc-65lzg: secret "image-registry-tls" not found
Jul 02 19:06:01.462063 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:01.461795    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls podName:a3d9e565-fc54-4f19-b4f4-bf30cb043466 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:09.461780204 +0000 UTC m=+50.611812166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls") pod "image-registry-64447cc5fc-65lzg" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466") : secret "image-registry-tls" not found
Jul 02 19:06:01.462063 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:01.461813    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls podName:99f84018-e5e1-4d3e-8b4d-7c96d34480d3 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:09.461803284 +0000 UTC m=+50.611835251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls") pod "dns-default-dmlds" (UID: "99f84018-e5e1-4d3e-8b4d-7c96d34480d3") : secret "dns-default-metrics-tls" not found
Jul 02 19:06:04.428838 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.428806    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hg6fm"]
Jul 02 19:06:04.460545 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.460512    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hg6fm"]
Jul 02 19:06:04.460707 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.460632    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.463991 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.463948    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 19:06:04.463991 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.463957    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 19:06:04.465120 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.465101    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sbf27\""
Jul 02 19:06:04.465236 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.465126    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 19:06:04.465236 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.465186    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 19:06:04.586359 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.586322    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.586359 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.586357    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.586581 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.586383    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr952\" (UniqueName: \"kubernetes.io/projected/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-api-access-qr952\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.586581 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.586405    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-data-volume\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.586581 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.586499    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-crio-socket\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.687830 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.687755    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-data-volume\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.687830 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.687812    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-crio-socket\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.687969 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.687954    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-crio-socket\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.688005 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.687979    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.688041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.688005    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.688041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.688033    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr952\" (UniqueName: \"kubernetes.io/projected/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-api-access-qr952\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.688130 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.688107    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-data-volume\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:04.688189 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:04.688119    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:06:04.688226 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:04.688206    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:04.688259 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:04.688253    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:05.18822917 +0000 UTC m=+46.338261150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : configmap "kube-rbac-proxy" not found
Jul 02 19:06:04.688295 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:04.688267    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:05.188260916 +0000 UTC m=+46.338292877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:04.698585 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:04.698560    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr952\" (UniqueName: \"kubernetes.io/projected/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-api-access-qr952\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:05.193157 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:05.193118    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:05.193341 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:05.193176    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:05.193341 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:05.193259    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:06:05.193413 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:05.193345    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:06.19332335 +0000 UTC m=+47.343355716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : configmap "kube-rbac-proxy" not found
Jul 02 19:06:05.193413 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:05.193361    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:05.193413 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:05.193399    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:06.193388601 +0000 UTC m=+47.343420562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:06.200623 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:06.200590    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:06.200623 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:06.200629    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:06.201053 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:06.200708    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:06:06.201053 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:06.200735    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:06.201053 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:06.200787    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:08.200770998 +0000 UTC m=+49.350802962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : configmap "kube-rbac-proxy" not found
Jul 02 19:06:06.201053 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:06.200801    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:08.200795035 +0000 UTC m=+49.350826996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:06.602665 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:06.602583    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xztmn_bd214edd-b8f8-4814-accb-dd631b0f402d/dns-node-resolver/0.log"
Jul 02 19:06:07.603847 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:07.603821    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kjg2x_26ec4b82-e24c-4d87-b690-91add3af40c4/node-ca/0.log"
Jul 02 19:06:08.207380 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:08.207350    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jwx7v_36604680-9a15-4463-aaa5-721782988b0a/serve-healthcheck-canary/0.log"
Jul 02 19:06:08.219615 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:08.219589    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:08.219691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:08.219623    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:08.219739 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:08.219710    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:06:08.219781 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:08.219768    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:08.219815 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:08.219781    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:12.219765351 +0000 UTC m=+53.369797311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : configmap "kube-rbac-proxy" not found
Jul 02 19:06:08.219815 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:08.219815    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:12.219804301 +0000 UTC m=+53.369836261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:09.528529 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:09.528486    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:09.528912 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:09.528562    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:09.528912 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:09.528644    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:06:09.528912 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:09.528663    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64447cc5fc-65lzg: secret "image-registry-tls" not found
Jul 02 19:06:09.528912 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:09.528668    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:06:09.528912 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:09.528716    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls podName:a3d9e565-fc54-4f19-b4f4-bf30cb043466 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:25.5287005 +0000 UTC m=+66.678732460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls") pod "image-registry-64447cc5fc-65lzg" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466") : secret "image-registry-tls" not found
Jul 02 19:06:09.528912 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:09.528730    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls podName:99f84018-e5e1-4d3e-8b4d-7c96d34480d3 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:25.528723812 +0000 UTC m=+66.678755773 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls") pod "dns-default-dmlds" (UID: "99f84018-e5e1-4d3e-8b4d-7c96d34480d3") : secret "dns-default-metrics-tls" not found
Jul 02 19:06:12.251300 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:12.251261    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:12.251300 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:12.251303    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:12.251797 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:12.251416    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:12.251797 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:12.251478    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls podName:4a6fd1fa-b6de-4952-b0e6-b5d4ef848373 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:20.251463804 +0000 UTC m=+61.401495766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hg6fm" (UID: "4a6fd1fa-b6de-4952-b0e6-b5d4ef848373") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:06:12.251868 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:12.251813    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:20.316553 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.316516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:20.318912 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.318882    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a6fd1fa-b6de-4952-b0e6-b5d4ef848373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hg6fm\" (UID: \"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373\") " pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:20.371507 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.371481    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sbf27\""
Jul 02 19:06:20.379775 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.379751    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hg6fm"
Jul 02 19:06:20.497605 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.497584    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgwpk"
Jul 02 19:06:20.542595 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.542562    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hg6fm"]
Jul 02 19:06:20.546604 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:20.546576    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a6fd1fa_b6de_4952_b0e6_b5d4ef848373.slice/crio-1f2a5650afcca61c18e9c8c8f404d9934ff1d86ae3f306d172223c121cb75f0b WatchSource:0}: Error finding container 1f2a5650afcca61c18e9c8c8f404d9934ff1d86ae3f306d172223c121cb75f0b: Status 404 returned error can't find the container with id 1f2a5650afcca61c18e9c8c8f404d9934ff1d86ae3f306d172223c121cb75f0b
Jul 02 19:06:20.565798 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:20.565765    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hg6fm" event={"ID":"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373","Type":"ContainerStarted","Data":"1f2a5650afcca61c18e9c8c8f404d9934ff1d86ae3f306d172223c121cb75f0b"}
Jul 02 19:06:24.578831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:24.578796    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hg6fm" event={"ID":"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373","Type":"ContainerStarted","Data":"89fb915a5d64de0ff192fafd9c97be6820e1c8d8175bd81a6f6505a87e9d0a69"}
Jul 02 19:06:25.059561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.059522    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:06:25.059767 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.059578    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:06:25.062487 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.062459    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 19:06:25.062602 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.062459    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 19:06:25.073317 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.073256    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e757a13e-15b8-4415-bcf3-acc95b235a71-metrics-certs\") pod \"network-metrics-daemon-crgh4\" (UID: \"e757a13e-15b8-4415-bcf3-acc95b235a71\") " pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:06:25.073458 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.073344    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/575a34d7-8a37-4869-8082-5383d427e35b-original-pull-secret\") pod \"global-pull-secret-syncer-fc92r\" (UID: \"575a34d7-8a37-4869-8082-5383d427e35b\") " pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:06:25.160822 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.160787    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:06:25.163717 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.163686    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 19:06:25.174362 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.174343    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 19:06:25.183471 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.183447    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx8tr\" (UniqueName: \"kubernetes.io/projected/10c62fd6-7471-43e6-9036-f7351746d419-kube-api-access-gx8tr\") pod \"network-check-target-vrgc9\" (UID: \"10c62fd6-7471-43e6-9036-f7351746d419\") " pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:06:25.221722 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.221686    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fc92r"
Jul 02 19:06:25.231022 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.230994    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjgvz\""
Jul 02 19:06:25.236697 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.236677    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mxlv5\""
Jul 02 19:06:25.238717 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.238702    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-crgh4"
Jul 02 19:06:25.244922 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.244895    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:06:25.363292 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.363259    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fc92r"]
Jul 02 19:06:25.372869 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:25.372834    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575a34d7_8a37_4869_8082_5383d427e35b.slice/crio-d0725fcd88c73e19e8613bf8727b921dd49cc3daad1ddffc59cf402189df5653 WatchSource:0}: Error finding container d0725fcd88c73e19e8613bf8727b921dd49cc3daad1ddffc59cf402189df5653: Status 404 returned error can't find the container with id d0725fcd88c73e19e8613bf8727b921dd49cc3daad1ddffc59cf402189df5653
Jul 02 19:06:25.381139 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.381114    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-crgh4"]
Jul 02 19:06:25.384007 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:25.383981    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode757a13e_15b8_4415_bcf3_acc95b235a71.slice/crio-65f005e66094b6ab455ac4158885f89225ada61ffa3dcff6154df326f73b75ad WatchSource:0}: Error finding container 65f005e66094b6ab455ac4158885f89225ada61ffa3dcff6154df326f73b75ad: Status 404 returned error can't find the container with id 65f005e66094b6ab455ac4158885f89225ada61ffa3dcff6154df326f73b75ad
Jul 02 19:06:25.398647 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.398619    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vrgc9"]
Jul 02 19:06:25.401415 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:25.401395    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c62fd6_7471_43e6_9036_f7351746d419.slice/crio-d2e68e5ee0457661aa24d37212d720c32b96503f1e2a9577f6bd3c3e9cfd6021 WatchSource:0}: Error finding container d2e68e5ee0457661aa24d37212d720c32b96503f1e2a9577f6bd3c3e9cfd6021: Status 404 returned error can't find the container with id d2e68e5ee0457661aa24d37212d720c32b96503f1e2a9577f6bd3c3e9cfd6021
Jul 02 19:06:25.563286 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.563202    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:25.563286 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.563263    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:25.565610 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.565586    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99f84018-e5e1-4d3e-8b4d-7c96d34480d3-metrics-tls\") pod \"dns-default-dmlds\" (UID: \"99f84018-e5e1-4d3e-8b4d-7c96d34480d3\") " pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:25.565719 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.565624    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"image-registry-64447cc5fc-65lzg\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") " pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:25.583294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.583259    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hg6fm" event={"ID":"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373","Type":"ContainerStarted","Data":"c328b8913ca0d354d5cdc67ab5ced71d476c6d7ce070668ad8c0e6b1af7b73d8"}
Jul 02 19:06:25.584358 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.584338    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-crgh4" event={"ID":"e757a13e-15b8-4415-bcf3-acc95b235a71","Type":"ContainerStarted","Data":"65f005e66094b6ab455ac4158885f89225ada61ffa3dcff6154df326f73b75ad"}
Jul 02 19:06:25.585433 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.585409    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fc92r" event={"ID":"575a34d7-8a37-4869-8082-5383d427e35b","Type":"ContainerStarted","Data":"d0725fcd88c73e19e8613bf8727b921dd49cc3daad1ddffc59cf402189df5653"}
Jul 02 19:06:25.586362 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.586341    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vrgc9" event={"ID":"10c62fd6-7471-43e6-9036-f7351746d419","Type":"ContainerStarted","Data":"d2e68e5ee0457661aa24d37212d720c32b96503f1e2a9577f6bd3c3e9cfd6021"}
Jul 02 19:06:25.836939 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.836855    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-68mj4\""
Jul 02 19:06:25.843352 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.843320    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:25.843575 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.843536    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crw8l\""
Jul 02 19:06:25.851344 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:25.851315    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:26.014955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.014877    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64447cc5fc-65lzg"]
Jul 02 19:06:26.018025 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.017087    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"]
Jul 02 19:06:26.021505 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.020890    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.035368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.033428    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 02 19:06:26.035368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.033830    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 02 19:06:26.035368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.034633    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 19:06:26.035368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.034835    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 02 19:06:26.035368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.035030    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 19:06:26.035368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.035238    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 02 19:06:26.037933 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.037692    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 19:06:26.050315 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.050252    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"]
Jul 02 19:06:26.051931 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.051854    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64447cc5fc-65lzg"]
Jul 02 19:06:26.053740 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:26.053706    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d9e565_fc54_4f19_b4f4_bf30cb043466.slice/crio-c3e2be52c5bdc7ed8092a1d065dea0bf8a3fce396345e878b8172e6f70fce3d5 WatchSource:0}: Error finding container c3e2be52c5bdc7ed8092a1d065dea0bf8a3fce396345e878b8172e6f70fce3d5: Status 404 returned error can't find the container with id c3e2be52c5bdc7ed8092a1d065dea0bf8a3fce396345e878b8172e6f70fce3d5
Jul 02 19:06:26.068715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.068259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-ca\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.068715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.068302    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.068715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.068332    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv89c\" (UniqueName: \"kubernetes.io/projected/b67f309c-809a-4d78-8b6d-42daa1b06834-kube-api-access-sv89c\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.068715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.068369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.068715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.068397    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-hub\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.068715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.068423    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67f309c-809a-4d78-8b6d-42daa1b06834-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.072374 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.072166    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dmlds"]
Jul 02 19:06:26.135089 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.134786    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-548595b9dc-ct24l"]
Jul 02 19:06:26.138103 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.138079    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.169480 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169405    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-548595b9dc-ct24l"]
Jul 02 19:06:26.169651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169511    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-ca\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.169651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169543    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.169651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169568    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv89c\" (UniqueName: \"kubernetes.io/projected/b67f309c-809a-4d78-8b6d-42daa1b06834-kube-api-access-sv89c\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.169651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169603    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.169651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169629    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-hub\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.169651 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.169652    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67f309c-809a-4d78-8b6d-42daa1b06834-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.173367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.173289    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67f309c-809a-4d78-8b6d-42daa1b06834-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.177269 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.177241    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.178698 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.178416    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.180004 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.179989    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-ca\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.180995 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.180979    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b67f309c-809a-4d78-8b6d-42daa1b06834-hub\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.212254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.212191    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv89c\" (UniqueName: \"kubernetes.io/projected/b67f309c-809a-4d78-8b6d-42daa1b06834-kube-api-access-sv89c\") pod \"cluster-proxy-proxy-agent-7c44447b5f-9q6dn\" (UID: \"b67f309c-809a-4d78-8b6d-42daa1b06834\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.270728 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270684    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-registry-tls\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.270906 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270745    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-installation-pull-secrets\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.270906 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270787    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hjft\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-kube-api-access-5hjft\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.270906 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270818    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-trusted-ca\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.270906 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270867    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-bound-sa-token\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.271196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270920    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-ca-trust-extracted\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.271196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270945    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-registry-certificates\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.271196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.270971    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-image-registry-private-configuration\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.343080 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.342866    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.371946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-ca-trust-extracted\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.371995    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-registry-certificates\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372025    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-image-registry-private-configuration\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372056    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-registry-tls\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372084    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-installation-pull-secrets\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372122    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hjft\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-kube-api-access-5hjft\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372164    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-trusted-ca\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372212    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-bound-sa-token\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.372977    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-ca-trust-extracted\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.373970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.373784    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-registry-certificates\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.375507 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.375463    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-trusted-ca\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.378679 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.378620    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-installation-pull-secrets\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.381694 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.381611    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-registry-tls\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.387031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.386964    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-image-registry-private-configuration\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.400197 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.399583    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-bound-sa-token\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.400197 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.400128    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hjft\" (UniqueName: \"kubernetes.io/projected/6e74e3a7-c2b7-466e-9c57-c0e9126e4146-kube-api-access-5hjft\") pod \"image-registry-548595b9dc-ct24l\" (UID: \"6e74e3a7-c2b7-466e-9c57-c0e9126e4146\") " pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.450710 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.450440    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:26.592384 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.592239    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dmlds" event={"ID":"99f84018-e5e1-4d3e-8b4d-7c96d34480d3","Type":"ContainerStarted","Data":"fce51ee68eac02546248dc457d5c8f2e204b90ed2fa4fb8ba936886572ea92ef"}
Jul 02 19:06:26.595219 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.594642    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" event={"ID":"a3d9e565-fc54-4f19-b4f4-bf30cb043466","Type":"ContainerStarted","Data":"812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3"}
Jul 02 19:06:26.595219 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.594700    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" event={"ID":"a3d9e565-fc54-4f19-b4f4-bf30cb043466","Type":"ContainerStarted","Data":"c3e2be52c5bdc7ed8092a1d065dea0bf8a3fce396345e878b8172e6f70fce3d5"}
Jul 02 19:06:26.595219 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.595068    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:26.889262 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.889201    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" podStartSLOduration=53.889176323 podStartE2EDuration="53.889176323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:26.614361545 +0000 UTC m=+67.764393529" watchObservedRunningTime="2026-07-02 19:06:26.889176323 +0000 UTC m=+68.039208307"
Jul 02 19:06:26.889803 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.889780    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn"]
Jul 02 19:06:26.914888 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:26.914638    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-548595b9dc-ct24l"]
Jul 02 19:06:26.925096 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:26.925067    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e74e3a7_c2b7_466e_9c57_c0e9126e4146.slice/crio-0bd07f14eaec4132ebde17676e431c92e8df5381bfa4c7d230a3e37b875a11bd WatchSource:0}: Error finding container 0bd07f14eaec4132ebde17676e431c92e8df5381bfa4c7d230a3e37b875a11bd: Status 404 returned error can't find the container with id 0bd07f14eaec4132ebde17676e431c92e8df5381bfa4c7d230a3e37b875a11bd
Jul 02 19:06:27.603067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.602983    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-crgh4" event={"ID":"e757a13e-15b8-4415-bcf3-acc95b235a71","Type":"ContainerStarted","Data":"2f3f551596e8402f664a6c5dedf5ce4fac5194a4ed1493971af514ada528b2db"}
Jul 02 19:06:27.603067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.603052    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-crgh4" event={"ID":"e757a13e-15b8-4415-bcf3-acc95b235a71","Type":"ContainerStarted","Data":"0e5d39f7c3450edd27f18b1ba430820b72be1c6a6762705373c73d0be7881edc"}
Jul 02 19:06:27.608041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.606627    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548595b9dc-ct24l" event={"ID":"6e74e3a7-c2b7-466e-9c57-c0e9126e4146","Type":"ContainerStarted","Data":"05b84679f7fd906333aa47616a5f46791ca4c1bc35198a1287fea1ff87e77f93"}
Jul 02 19:06:27.608041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.606678    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548595b9dc-ct24l" event={"ID":"6e74e3a7-c2b7-466e-9c57-c0e9126e4146","Type":"ContainerStarted","Data":"0bd07f14eaec4132ebde17676e431c92e8df5381bfa4c7d230a3e37b875a11bd"}
Jul 02 19:06:27.608041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.607187    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:27.613083 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.613022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn" event={"ID":"b67f309c-809a-4d78-8b6d-42daa1b06834","Type":"ContainerStarted","Data":"69fc8888ffb18704e482debb15b7bf601454c075e626af5cbcb22d9e429e8833"}
Jul 02 19:06:27.624134 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:27.623117    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-crgh4" podStartSLOduration=67.235807164 podStartE2EDuration="1m8.623097151s" totalImagesPullingTime="1.387289987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:27.621494032 +0000 UTC m=+68.771526029" watchObservedRunningTime="2026-07-02 19:06:27.623097151 +0000 UTC m=+68.773129140"
Jul 02 19:06:34.645097 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.645040    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn" event={"ID":"b67f309c-809a-4d78-8b6d-42daa1b06834","Type":"ContainerStarted","Data":"265a5e30bf4aa9abb228d24daad10a91149d99144a9dbc34d4b6a5331d578431"}
Jul 02 19:06:34.645097 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.645097    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn" event={"ID":"b67f309c-809a-4d78-8b6d-42daa1b06834","Type":"ContainerStarted","Data":"ab44bbd7712b52e72d9a3979dddc12e075a47fcc3abc72cd0c37bf3afd8b9d46"}
Jul 02 19:06:34.645097 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.645113    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn" event={"ID":"b67f309c-809a-4d78-8b6d-42daa1b06834","Type":"ContainerStarted","Data":"50c1a0e0f749733915f47e91cc28072c58964fbb7d45fdba7998acd438e1e59c"}
Jul 02 19:06:34.647139 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.647115    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hg6fm" event={"ID":"4a6fd1fa-b6de-4952-b0e6-b5d4ef848373","Type":"ContainerStarted","Data":"ccac487f5cfd06e564a1ade9c274436f70291fdb5746733a68ddc9c4b9587850"}
Jul 02 19:06:34.648506 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.648484    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fc92r" event={"ID":"575a34d7-8a37-4869-8082-5383d427e35b","Type":"ContainerStarted","Data":"62c04a75c3a84a317baed5d998ca68d8c0eaf4dce2369795bc8485d93104d226"}
Jul 02 19:06:34.650277 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.650235    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vrgc9" event={"ID":"10c62fd6-7471-43e6-9036-f7351746d419","Type":"ContainerStarted","Data":"3ef98cd0ace2ee596acc2ac115020686449274deafbf2fa49d9520efc495c7ca"}
Jul 02 19:06:34.650654 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.650611    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:06:34.652453 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.652434    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dmlds" event={"ID":"99f84018-e5e1-4d3e-8b4d-7c96d34480d3","Type":"ContainerStarted","Data":"5c9c62135be1393fba5f6a8a1137e3cd7172cd454c03af0905204cc4bba7e75a"}
Jul 02 19:06:34.652545 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.652489    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dmlds" event={"ID":"99f84018-e5e1-4d3e-8b4d-7c96d34480d3","Type":"ContainerStarted","Data":"be688b9ce78ba08383152accf874eff2c597e223fd84c01af11b04bf7f1afffc"}
Jul 02 19:06:34.652545 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.652527    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:34.669089 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.669045    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c44447b5f-9q6dn" podStartSLOduration=2.185930378 podStartE2EDuration="9.669031622s" totalImagesPullingTime="7.483101244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:34.667421102 +0000 UTC m=+75.817453085" watchObservedRunningTime="2026-07-02 19:06:34.669031622 +0000 UTC m=+75.819063605"
Jul 02 19:06:34.669231 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.669211    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-548595b9dc-ct24l" podStartSLOduration=8.669202595 podStartE2EDuration="8.669202595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:27.646323767 +0000 UTC m=+68.796355750" watchObservedRunningTime="2026-07-02 19:06:34.669202595 +0000 UTC m=+75.819234577"
Jul 02 19:06:34.688350 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.688302    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dmlds" podStartSLOduration=33.389662996 podStartE2EDuration="41.688288295s" totalImagesPullingTime="8.298625299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:34.688075957 +0000 UTC m=+75.838107967" watchObservedRunningTime="2026-07-02 19:06:34.688288295 +0000 UTC m=+75.838320277"
Jul 02 19:06:34.712411 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.712369    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hg6fm" podStartSLOduration=16.999046087 podStartE2EDuration="30.712356042s" totalImagesPullingTime="13.713309955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:06:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:34.711193602 +0000 UTC m=+75.861225587" watchObservedRunningTime="2026-07-02 19:06:34.712356042 +0000 UTC m=+75.862388025"
Jul 02 19:06:34.728409 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.728366    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vrgc9" podStartSLOduration=66.742285781 podStartE2EDuration="1m15.72835448s" totalImagesPullingTime="8.986068699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:34.727545327 +0000 UTC m=+75.877577307" watchObservedRunningTime="2026-07-02 19:06:34.72835448 +0000 UTC m=+75.878386464"
Jul 02 19:06:34.747288 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:34.747248    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fc92r" podStartSLOduration=66.737095098 podStartE2EDuration="1m15.74723545s" totalImagesPullingTime="9.010140352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:34.74638558 +0000 UTC m=+75.896417566" watchObservedRunningTime="2026-07-02 19:06:34.74723545 +0000 UTC m=+75.897267432"
Jul 02 19:06:40.667918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.667886    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-99r8p"]
Jul 02 19:06:40.673081 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.673056    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.675794 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.675768    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 19:06:40.676035 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.676014    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 19:06:40.676035 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.676029    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 19:06:40.676198 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.676026    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 19:06:40.676198 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.676104    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p6znq\""
Jul 02 19:06:40.676998 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.676982    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 19:06:40.677087 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.677055    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 19:06:40.780355 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780315    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-accelerators-collector-config\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780355 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780353    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2924z\" (UniqueName: \"kubernetes.io/projected/0ecadfce-a9e6-4e71-8964-d96b54d4a382-kube-api-access-2924z\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780380    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-wtmp\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780397    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-textfile\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780474    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-sys\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780496    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780523    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-tls\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ecadfce-a9e6-4e71-8964-d96b54d4a382-metrics-client-ca\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.780561 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.780558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-root\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881042 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881005    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-root\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881042 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881043    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-accelerators-collector-config\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881069    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2924z\" (UniqueName: \"kubernetes.io/projected/0ecadfce-a9e6-4e71-8964-d96b54d4a382-kube-api-access-2924z\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881103    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-wtmp\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881118    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-root\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881128    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-textfile\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-sys\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881264    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-sys\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881264    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-wtmp\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881340    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-tls\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881374    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ecadfce-a9e6-4e71-8964-d96b54d4a382-metrics-client-ca\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.881691 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:40.881444    2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 02 19:06:40.881897 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881873    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-textfile\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.882062 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:40.881980    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-tls podName:0ecadfce-a9e6-4e71-8964-d96b54d4a382 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:41.381954188 +0000 UTC m=+82.531986153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-tls") pod "node-exporter-99r8p" (UID: "0ecadfce-a9e6-4e71-8964-d96b54d4a382") : secret "node-exporter-tls" not found
Jul 02 19:06:40.882190 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.881991    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-accelerators-collector-config\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.882497 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.882475    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ecadfce-a9e6-4e71-8964-d96b54d4a382-metrics-client-ca\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.884592 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.884565    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:40.890811 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:40.890792    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2924z\" (UniqueName: \"kubernetes.io/projected/0ecadfce-a9e6-4e71-8964-d96b54d4a382-kube-api-access-2924z\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:41.384209 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:41.384170    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-tls\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:41.386350 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:41.386323    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ecadfce-a9e6-4e71-8964-d96b54d4a382-node-exporter-tls\") pod \"node-exporter-99r8p\" (UID: \"0ecadfce-a9e6-4e71-8964-d96b54d4a382\") " pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:41.582823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:41.582790    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-99r8p"
Jul 02 19:06:41.593951 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:41.593923    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecadfce_a9e6_4e71_8964_d96b54d4a382.slice/crio-0974373958aea97662468a9e7087374118d43ad9470e30ea737c4b0957e1d799 WatchSource:0}: Error finding container 0974373958aea97662468a9e7087374118d43ad9470e30ea737c4b0957e1d799: Status 404 returned error can't find the container with id 0974373958aea97662468a9e7087374118d43ad9470e30ea737c4b0957e1d799
Jul 02 19:06:41.679035 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:41.679001    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99r8p" event={"ID":"0ecadfce-a9e6-4e71-8964-d96b54d4a382","Type":"ContainerStarted","Data":"0974373958aea97662468a9e7087374118d43ad9470e30ea737c4b0957e1d799"}
Jul 02 19:06:43.686012 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:43.685979    2585 generic.go:356] "Generic (PLEG): container finished" podID="0ecadfce-a9e6-4e71-8964-d96b54d4a382" containerID="af3bfac415da8cf65796669e2a935b4e88761c2cda0dfe3039d28f9cb96b64bf" exitCode=0
Jul 02 19:06:43.686423 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:43.686036    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99r8p" event={"ID":"0ecadfce-a9e6-4e71-8964-d96b54d4a382","Type":"ContainerDied","Data":"af3bfac415da8cf65796669e2a935b4e88761c2cda0dfe3039d28f9cb96b64bf"}
Jul 02 19:06:44.658264 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.658232    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dmlds"
Jul 02 19:06:44.692230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.692188    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99r8p" event={"ID":"0ecadfce-a9e6-4e71-8964-d96b54d4a382","Type":"ContainerStarted","Data":"8711db19e43e5677cb498dcc35498b2418470ae2df3d24a07071ef6631c5968c"}
Jul 02 19:06:44.692683 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.692266    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99r8p" event={"ID":"0ecadfce-a9e6-4e71-8964-d96b54d4a382","Type":"ContainerStarted","Data":"0a3c125a324c8acb14f2da08941fc4482607759d4bd14d69682bc015f668f145"}
Jul 02 19:06:44.713520 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.713465    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-99r8p" podStartSLOduration=3.608667386 podStartE2EDuration="4.713448495s" totalImagesPullingTime="1.045469744s" totalInitContainerRuntime="59.311365ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:44.712410239 +0000 UTC m=+85.862442221" watchObservedRunningTime="2026-07-02 19:06:44.713448495 +0000 UTC m=+85.863480477"
Jul 02 19:06:44.918415 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.918336    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6b84dd98f9-clmdm"]
Jul 02 19:06:44.921380 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.921365    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:44.928061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.925641    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tzc2v\""
Jul 02 19:06:44.928061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.925881    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 02 19:06:44.928061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.925916    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 02 19:06:44.928061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.925644    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 02 19:06:44.928061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.926483    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4giomdlbgrpmu\""
Jul 02 19:06:44.928061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.926708    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 02 19:06:44.932841 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:44.932818    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b84dd98f9-clmdm"]
Jul 02 19:06:45.011488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011452    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29cd2afc-b74c-469e-b131-0e561e552253-audit-log\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.011488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011490    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29cd2afc-b74c-469e-b131-0e561e552253-metrics-server-audit-profiles\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.011704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011528    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29cd2afc-b74c-469e-b131-0e561e552253-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.011704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011600    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-secret-metrics-server-tls\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.011704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011642    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-secret-metrics-server-client-certs\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.011704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011660    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-client-ca-bundle\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.011704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.011688    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghsgj\" (UniqueName: \"kubernetes.io/projected/29cd2afc-b74c-469e-b131-0e561e552253-kube-api-access-ghsgj\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112619 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112580    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-secret-metrics-server-tls\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112619 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112625    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-secret-metrics-server-client-certs\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112820 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112643    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-client-ca-bundle\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112820 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112665    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghsgj\" (UniqueName: \"kubernetes.io/projected/29cd2afc-b74c-469e-b131-0e561e552253-kube-api-access-ghsgj\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112820 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29cd2afc-b74c-469e-b131-0e561e552253-audit-log\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112820 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112711    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29cd2afc-b74c-469e-b131-0e561e552253-metrics-server-audit-profiles\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.112820 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.112736    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29cd2afc-b74c-469e-b131-0e561e552253-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.113235 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.113200    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29cd2afc-b74c-469e-b131-0e561e552253-audit-log\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.113557 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.113535    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29cd2afc-b74c-469e-b131-0e561e552253-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.113789 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.113770    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29cd2afc-b74c-469e-b131-0e561e552253-metrics-server-audit-profiles\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.115230 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.115200    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-secret-metrics-server-tls\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.115671 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.115650    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-client-ca-bundle\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.115711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.115693    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29cd2afc-b74c-469e-b131-0e561e552253-secret-metrics-server-client-certs\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.120563 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.120542    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghsgj\" (UniqueName: \"kubernetes.io/projected/29cd2afc-b74c-469e-b131-0e561e552253-kube-api-access-ghsgj\") pod \"metrics-server-6b84dd98f9-clmdm\" (UID: \"29cd2afc-b74c-469e-b131-0e561e552253\") " pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.234296 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.234206    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:06:45.359925 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.359789    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b84dd98f9-clmdm"]
Jul 02 19:06:45.363005 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:06:45.362977    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cd2afc_b74c_469e_b131_0e561e552253.slice/crio-d4082872f98489b7b05d23bfb13f2d04c228ae494aaa01766af71ca108d12f86 WatchSource:0}: Error finding container d4082872f98489b7b05d23bfb13f2d04c228ae494aaa01766af71ca108d12f86: Status 404 returned error can't find the container with id d4082872f98489b7b05d23bfb13f2d04c228ae494aaa01766af71ca108d12f86
Jul 02 19:06:45.696359 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:45.696328    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm" event={"ID":"29cd2afc-b74c-469e-b131-0e561e552253","Type":"ContainerStarted","Data":"d4082872f98489b7b05d23bfb13f2d04c228ae494aaa01766af71ca108d12f86"}
Jul 02 19:06:46.455564 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.455511    2585 patch_prober.go:28] interesting pod/image-registry-548595b9dc-ct24l container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 19:06:46.455736 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.455581    2585 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-548595b9dc-ct24l" podUID="6e74e3a7-c2b7-466e-9c57-c0e9126e4146" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 19:06:46.601361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.601302    2585 patch_prober.go:28] interesting pod/image-registry-64447cc5fc-65lzg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 19:06:46.601541 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.601365    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" podUID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 19:06:46.890519 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.890496    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:06:46.893956 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.893940    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:46.896747 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.896722    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 02 19:06:46.896888 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.896722    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 02 19:06:46.896888 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.896754    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 02 19:06:46.897051 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.897037    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bjstg\""
Jul 02 19:06:46.897118 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.897104    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 02 19:06:46.897226 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.897214    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 02 19:06:46.897850 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.897834    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3naeu67aof1pf\""
Jul 02 19:06:46.897957 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.897912    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 02 19:06:46.898026 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.897980    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 02 19:06:46.898227 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.898209    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 02 19:06:46.898227 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.898224    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 02 19:06:46.898353 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.898276    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 02 19:06:46.898659 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.898640    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 02 19:06:46.899076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.899060    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 02 19:06:46.899451 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.899435    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 02 19:06:46.901358 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.901339    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 02 19:06:46.922328 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:46.922292    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:06:47.028972 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.028929    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029179 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.028980    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029179 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029017    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029179 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029071    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029179 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029100    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029179 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029126    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029190    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029227    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029264    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029296    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029322    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029348    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029392    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-web-config\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029437    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029463    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rtf2\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-kube-api-access-7rtf2\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029547    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029575    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config-out\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029600    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.029879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.029627    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130499    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-web-config\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130545    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130583    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rtf2\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-kube-api-access-7rtf2\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130612    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config-out\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130667    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130723    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130748    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130773    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130828    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130854    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130896    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.130918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130917    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.131565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.131565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.130980    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.131565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.131006    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.131565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.131003    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.131565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.131032    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.132027 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.131799    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.132178 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.132102    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.133000 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.132681    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.133100 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.133027    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.133704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.133677    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.135018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.134165    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.135018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.134414    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.135018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.134970    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.135018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.135012    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-web-config\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.135303 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.135202    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137053 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137029    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137053 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137042    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137234 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137206    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137296 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137265    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137627 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137603    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137689 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137676    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config-out\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.137851 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.137836    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.138916 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.138892    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.142566 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.142546    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rtf2\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-kube-api-access-7rtf2\") pod \"prometheus-k8s-0\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.204608 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.204567    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:06:47.367563 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.365993    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:06:47.705683 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.705648    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm" event={"ID":"29cd2afc-b74c-469e-b131-0e561e552253","Type":"ContainerStarted","Data":"d60843be26a2933c8cdedf3c3ef087e74e775a46cb8c9dc564ee0fbd07d25a46"}
Jul 02 19:06:47.706911 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.706885    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"3fd82a9078ba50d36ae26087a504874b3ab9974ebaa17023818b707c171bb21a"}
Jul 02 19:06:47.723987 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:47.723943    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm" podStartSLOduration=2.260666825 podStartE2EDuration="3.723931104s" totalImagesPullingTime="1.463264279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:47.723533248 +0000 UTC m=+88.873565232" watchObservedRunningTime="2026-07-02 19:06:47.723931104 +0000 UTC m=+88.873963088"
Jul 02 19:06:48.712306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:48.712274    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4" exitCode=0
Jul 02 19:06:48.712676 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:48.712358    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"}
Jul 02 19:06:49.627399 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:49.627365    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-548595b9dc-ct24l"
Jul 02 19:06:51.646386 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:51.646316    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" podUID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" containerName="registry" containerID="cri-o://812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3" gracePeriod=30
Jul 02 19:06:52.139261 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.139238    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:52.282694 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282592    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-bound-sa-token\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.282694 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282682    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-certificates\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.282910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282714    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hjgb\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-kube-api-access-5hjgb\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.282910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282740    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.282910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282765    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-trusted-ca\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.282910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282829    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-image-registry-private-configuration\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.282910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.282895    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-installation-pull-secrets\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.283268 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.283078    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d9e565-fc54-4f19-b4f4-bf30cb043466-ca-trust-extracted\") pod \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\" (UID: \"a3d9e565-fc54-4f19-b4f4-bf30cb043466\") "
Jul 02 19:06:52.290923 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.288530    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:06:52.290923 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.290829    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:06:52.292101 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.291997    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:06:52.292101 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.292080    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:06:52.292375 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.292224    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:06:52.292375 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.292325    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:06:52.294195 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.294139    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-kube-api-access-5hjgb" (OuterVolumeSpecName: "kube-api-access-5hjgb") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "kube-api-access-5hjgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:06:52.298306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.296181    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d9e565-fc54-4f19-b4f4-bf30cb043466-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a3d9e565-fc54-4f19-b4f4-bf30cb043466" (UID: "a3d9e565-fc54-4f19-b4f4-bf30cb043466"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:06:52.384412 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384373    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d9e565-fc54-4f19-b4f4-bf30cb043466-ca-trust-extracted\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384412 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384405    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-bound-sa-token\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384412 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384415    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-certificates\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384412 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384424    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hjgb\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-kube-api-access-5hjgb\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384673 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384433    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d9e565-fc54-4f19-b4f4-bf30cb043466-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384673 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384445    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d9e565-fc54-4f19-b4f4-bf30cb043466-trusted-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384673 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384458    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-image-registry-private-configuration\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.384673 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.384466    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d9e565-fc54-4f19-b4f4-bf30cb043466-installation-pull-secrets\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:06:52.729598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.729562    2585 generic.go:356] "Generic (PLEG): container finished" podID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" containerID="812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3" exitCode=0
Jul 02 19:06:52.730062 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.729610    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg"
Jul 02 19:06:52.730062 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.729659    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" event={"ID":"a3d9e565-fc54-4f19-b4f4-bf30cb043466","Type":"ContainerDied","Data":"812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3"}
Jul 02 19:06:52.730062 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.729723    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64447cc5fc-65lzg" event={"ID":"a3d9e565-fc54-4f19-b4f4-bf30cb043466","Type":"ContainerDied","Data":"c3e2be52c5bdc7ed8092a1d065dea0bf8a3fce396345e878b8172e6f70fce3d5"}
Jul 02 19:06:52.730062 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.729752    2585 scope.go:117] "RemoveContainer" containerID="812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3"
Jul 02 19:06:52.732863 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.732837    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"}
Jul 02 19:06:52.732969 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.732905    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"}
Jul 02 19:06:52.739388 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.739370    2585 scope.go:117] "RemoveContainer" containerID="812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3"
Jul 02 19:06:52.739657 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:06:52.739631    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3\": container with ID starting with 812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3 not found: ID does not exist" containerID="812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3"
Jul 02 19:06:52.739722 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.739670    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3"} err="failed to get container status \"812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3\": rpc error: code = NotFound desc = could not find container \"812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3\": container with ID starting with 812a72016b753fab218b51f59ee7d977f77be6594e1f43db45d3bc60ece5f4c3 not found: ID does not exist"
Jul 02 19:06:52.752380 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.752348    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64447cc5fc-65lzg"]
Jul 02 19:06:52.758766 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:52.758730    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64447cc5fc-65lzg"]
Jul 02 19:06:53.314028 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:53.313951    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" path="/var/lib/kubelet/pods/a3d9e565-fc54-4f19-b4f4-bf30cb043466/volumes"
Jul 02 19:06:54.744913 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:54.744878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"}
Jul 02 19:06:54.745313 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:54.744944    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"}
Jul 02 19:06:54.745313 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:54.744975    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"}
Jul 02 19:06:54.745313 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:54.745006    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerStarted","Data":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"}
Jul 02 19:06:54.782921 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:54.782242    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.6641559360000002 podStartE2EDuration="8.782221719s" totalImagesPullingTime="6.09965734s" totalInitContainerRuntime="18.408443ms" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:54.782027983 +0000 UTC m=+95.932059967" watchObservedRunningTime="2026-07-02 19:06:54.782221719 +0000 UTC m=+95.932253704"
Jul 02 19:06:57.205537 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:06:57.205497    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:07:05.235038 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:05.235002    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:07:05.235461 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:05.235059    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:07:06.662527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:06.662495    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vrgc9"
Jul 02 19:07:25.240071 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:25.240041    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:07:25.243985 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:25.243961    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6b84dd98f9-clmdm"
Jul 02 19:07:47.205501 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:47.205460    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:07:47.223531 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:47.223504    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:07:47.938473 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:07:47.938450    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:05.274987 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.274944    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:08:05.275474 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.275286    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="prometheus" containerID="cri-o://d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801" gracePeriod=600
Jul 02 19:08:05.275474 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.275303    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy" containerID="cri-o://9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435" gracePeriod=600
Jul 02 19:08:05.275474 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.275344    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="thanos-sidecar" containerID="cri-o://330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a" gracePeriod=600
Jul 02 19:08:05.275474 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.275398    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="config-reloader" containerID="cri-o://6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79" gracePeriod=600
Jul 02 19:08:05.275474 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.275410    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-web" containerID="cri-o://5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0" gracePeriod=600
Jul 02 19:08:05.275474 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.275349    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78" gracePeriod=600
Jul 02 19:08:05.513358 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.513333    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:05.562383 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562315    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-metrics-client-certs\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562383 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562350    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-thanos-prometheus-http-client-file\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562383 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562375    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-web-config\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562402    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config-out\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562419    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-grpc-tls\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562437    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-0\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562461    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-tls-assets\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562485    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-metrics-client-ca\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562508    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562533    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-kubelet-serving-ca-bundle\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562574    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-kube-rbac-proxy\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.562640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562603    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562649    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rtf2\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-kube-api-access-7rtf2\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562674    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-2\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562698    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-trusted-ca-bundle\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562722    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-tls\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562746    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-1\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562774    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562812    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-db\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563161 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.562844    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-serving-certs-ca-bundle\") pod \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\" (UID: \"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e\") "
Jul 02 19:08:05.563571 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.563385    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.564246 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.563922    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.566460 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.565262    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.566460 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.565562    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.566460 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.565586    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.566460 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.565956    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.566460 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.566250    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:08:05.566780 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.566463    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:08:05.566780 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.566520    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.566780 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.566605    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-kube-api-access-7rtf2" (OuterVolumeSpecName: "kube-api-access-7rtf2") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "kube-api-access-7rtf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:08:05.567042 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.567021    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.567167 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.567049    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config-out" (OuterVolumeSpecName: "config-out") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:08:05.567259 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.567116    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.567519 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.567493    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:08:05.567673 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.567638    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.567977 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.567946    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config" (OuterVolumeSpecName: "config") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.568732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.568704    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.568809 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.568725    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.569003 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.568984    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.575918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.575895    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-web-config" (OuterVolumeSpecName: "web-config") pod "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" (UID: "7d5a3b63-770a-4dbe-a0a5-8799dc95e30e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:08:05.663776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663742    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.663776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663770    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.663776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663785    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663799    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-db\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663811    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663824    2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-metrics-client-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663836    2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663850    2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-web-config\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663860    2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config-out\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663871    2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-grpc-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663881    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663893    2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-tls-assets\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663905    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-metrics-client-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663916    2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-config\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663929    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663941    2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-kube-rbac-proxy\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663953    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663967    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rtf2\" (UniqueName: \"kubernetes.io/projected/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-kube-api-access-7rtf2\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663981    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.664040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.663996    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:08:05.985979 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.985945    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78" exitCode=0
Jul 02 19:08:05.985979 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.985974    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435" exitCode=0
Jul 02 19:08:05.985979 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.985985    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0" exitCode=0
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.985994    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a" exitCode=0
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986002    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79" exitCode=0
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986009    2585 generic.go:356] "Generic (PLEG): container finished" podID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801" exitCode=0
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986025    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"}
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986035    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986120    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"}
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986166    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"}
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986174    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986194    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"}
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986218    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"}
Jul 02 19:08:05.986238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986242    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"}
Jul 02 19:08:05.986669 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.986266    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d5a3b63-770a-4dbe-a0a5-8799dc95e30e","Type":"ContainerDied","Data":"3fd82a9078ba50d36ae26087a504874b3ab9974ebaa17023818b707c171bb21a"}
Jul 02 19:08:05.994337 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:05.994320    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.001226 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.001210    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.007440 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.007425    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.011857 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.011839    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:08:06.014698 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.014681    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.016295 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.016279    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:08:06.021214 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.021199    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.027889 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.027873    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.033950 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.033935    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.034221 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.034195    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.034321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.034228    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"} err="failed to get container status \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist"
Jul 02 19:08:06.034321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.034273    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.034543 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.034527    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.034582 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.034550    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"} err="failed to get container status \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist"
Jul 02 19:08:06.034582 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.034569    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.034804 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.034788    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.034863 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.034809    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"} err="failed to get container status \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist"
Jul 02 19:08:06.034863 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.034826    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.035055 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.035040    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.035109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035058    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"} err="failed to get container status \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist"
Jul 02 19:08:06.035109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035080    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.035322 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.035306    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.035365 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035325    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"} err="failed to get container status \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist"
Jul 02 19:08:06.035365 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035339    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.035550 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.035536    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.035591 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035554    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"} err="failed to get container status \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist"
Jul 02 19:08:06.035591 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035566    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.035775 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:08:06.035758    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.035814 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035778    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"} err="failed to get container status \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist"
Jul 02 19:08:06.035814 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035790    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.035955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035938    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"} err="failed to get container status \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist"
Jul 02 19:08:06.036008 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.035957    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.036117 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036095    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"} err="failed to get container status \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist"
Jul 02 19:08:06.036244 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036118    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.036353 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036337    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"} err="failed to get container status \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist"
Jul 02 19:08:06.036410 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036354    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.036550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036536    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"} err="failed to get container status \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist"
Jul 02 19:08:06.036550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036550    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.036775 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036758    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"} err="failed to get container status \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist"
Jul 02 19:08:06.036823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036776    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.036958 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036944    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"} err="failed to get container status \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist"
Jul 02 19:08:06.036999 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.036958    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.037166 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037131    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"} err="failed to get container status \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist"
Jul 02 19:08:06.037250 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037168    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.037380 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037367    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"} err="failed to get container status \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist"
Jul 02 19:08:06.037446 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037381    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.037583 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037565    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"} err="failed to get container status \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist"
Jul 02 19:08:06.037583 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037583    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.037761 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037747    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"} err="failed to get container status \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist"
Jul 02 19:08:06.037807 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037760    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.037936 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037920    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"} err="failed to get container status \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist"
Jul 02 19:08:06.037936 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.037935    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.038105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038090    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"} err="failed to get container status \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist"
Jul 02 19:08:06.038167 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038105    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.038312 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038297    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"} err="failed to get container status \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist"
Jul 02 19:08:06.038312 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038312    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.038488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038469    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"} err="failed to get container status \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist"
Jul 02 19:08:06.038529 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038488    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.038654 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038640    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"} err="failed to get container status \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist"
Jul 02 19:08:06.038654 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038653    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.038834 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038820    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"} err="failed to get container status \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist"
Jul 02 19:08:06.038871 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.038834    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.039047 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039030    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"} err="failed to get container status \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist"
Jul 02 19:08:06.039103 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039047    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.039293 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039275    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"} err="failed to get container status \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist"
Jul 02 19:08:06.039341 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039293    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.039507 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039479    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"} err="failed to get container status \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist"
Jul 02 19:08:06.039547 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039508    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.039686 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039671    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"} err="failed to get container status \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist"
Jul 02 19:08:06.039727 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039686    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.039876 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039859    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"} err="failed to get container status \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist"
Jul 02 19:08:06.039916 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.039877    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.040040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040024    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"} err="failed to get container status \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist"
Jul 02 19:08:06.040081 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040041    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.040277 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040258    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"} err="failed to get container status \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist"
Jul 02 19:08:06.040345 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040278    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.040502 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040487    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"} err="failed to get container status \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist"
Jul 02 19:08:06.040549 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040502    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.040696 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040680    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"} err="failed to get container status \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist"
Jul 02 19:08:06.040736 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040696    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.040882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040866    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"} err="failed to get container status \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist"
Jul 02 19:08:06.040882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.040882    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.041095 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041080    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"} err="failed to get container status \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist"
Jul 02 19:08:06.041095 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041094    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.041306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041289    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"} err="failed to get container status \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist"
Jul 02 19:08:06.041306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041305    2585 scope.go:117] "RemoveContainer" containerID="f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"
Jul 02 19:08:06.041499 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041484    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78"} err="failed to get container status \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": rpc error: code = NotFound desc = could not find container \"f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78\": container with ID starting with f0506ece436f16e530ba941d15106c6afbd9d6d554e71e4063481f08f6e38e78 not found: ID does not exist"
Jul 02 19:08:06.041499 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041499    2585 scope.go:117] "RemoveContainer" containerID="9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"
Jul 02 19:08:06.041686 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041672    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435"} err="failed to get container status \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": rpc error: code = NotFound desc = could not find container \"9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435\": container with ID starting with 9fee272aea01d56dbabc57f9f2fc7ad37dfc230ac4edb7ef15ccd3d8d969d435 not found: ID does not exist"
Jul 02 19:08:06.041686 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041685    2585 scope.go:117] "RemoveContainer" containerID="5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"
Jul 02 19:08:06.041886 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041870    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0"} err="failed to get container status \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": rpc error: code = NotFound desc = could not find container \"5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0\": container with ID starting with 5c7fa096a43439cb27c46d649ebad305ee5fc7d1f348ec1b255f5121b67b71a0 not found: ID does not exist"
Jul 02 19:08:06.041929 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.041885    2585 scope.go:117] "RemoveContainer" containerID="330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"
Jul 02 19:08:06.042125 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042107    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a"} err="failed to get container status \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": rpc error: code = NotFound desc = could not find container \"330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a\": container with ID starting with 330ec7e494fef8ccb4443dc1466ec8858ac642bcb8d0988a915be9567916cd6a not found: ID does not exist"
Jul 02 19:08:06.042183 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042126    2585 scope.go:117] "RemoveContainer" containerID="6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"
Jul 02 19:08:06.042333 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042318    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79"} err="failed to get container status \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": rpc error: code = NotFound desc = could not find container \"6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79\": container with ID starting with 6a505ca359e038b9d2ded555e733a0118d027744f050ef1385783ef24c42be79 not found: ID does not exist"
Jul 02 19:08:06.042372 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042333    2585 scope.go:117] "RemoveContainer" containerID="d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"
Jul 02 19:08:06.042507 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042492    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801"} err="failed to get container status \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": rpc error: code = NotFound desc = could not find container \"d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801\": container with ID starting with d0c63a824d8ce1f82087f71e71eee652653bac4361e5a91b0233542df6042801 not found: ID does not exist"
Jul 02 19:08:06.042507 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042506    2585 scope.go:117] "RemoveContainer" containerID="a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"
Jul 02 19:08:06.042690 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.042665    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4"} err="failed to get container status \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": rpc error: code = NotFound desc = could not find container \"a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4\": container with ID starting with a13e473dc5caa12b15153256ef0ae1b0198fa69432aa9718a398ef8ee88150c4 not found: ID does not exist"
Jul 02 19:08:06.053089 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053065    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:08:06.053339 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053324    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="init-config-reloader"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053342    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="init-config-reloader"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053353    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="config-reloader"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053362    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="config-reloader"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053373    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-thanos"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053381    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-thanos"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053393    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" containerName="registry"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053401    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" containerName="registry"
Jul 02 19:08:06.053418 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053414    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053422    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053436    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="prometheus"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053444    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="prometheus"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053463    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="thanos-sidecar"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053471    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="thanos-sidecar"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053478    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-web"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053487    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-web"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053543    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053556    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="config-reloader"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053566    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="thanos-sidecar"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053573    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-web"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053581    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="kube-rbac-proxy-thanos"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053590    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" containerName="prometheus"
Jul 02 19:08:06.053806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.053599    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d9e565-fc54-4f19-b4f4-bf30cb043466" containerName="registry"
Jul 02 19:08:06.058725 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.058708    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.062046 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062027    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 02 19:08:06.062214 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062194    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 02 19:08:06.062297 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062255    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3naeu67aof1pf\""
Jul 02 19:08:06.062297 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062273    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 02 19:08:06.062523 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062507    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 02 19:08:06.062607 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062590    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bjstg\""
Jul 02 19:08:06.062658 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062608    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 02 19:08:06.062658 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062645    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 02 19:08:06.062764 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062708    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 02 19:08:06.062831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062817    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 02 19:08:06.062910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.062894    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 02 19:08:06.063080 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.063054    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 02 19:08:06.063204 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.063184    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 02 19:08:06.063204 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.063193    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 02 19:08:06.066384 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066369    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 02 19:08:06.066629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066610    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066679 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066662    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066689    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066724    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-config\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066763    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066796    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066901 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066859    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066935 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066903    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.066979 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066939    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.066999    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067097 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067032    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntzrl\" (UniqueName: \"kubernetes.io/projected/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-kube-api-access-ntzrl\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067097 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067081    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067218 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067109    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067218 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067139    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067218 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067194    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067218    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067242    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067272    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067329    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.067499 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.067372    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.069714 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.069699    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 02 19:08:06.076938 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.076905    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:08:06.168353 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168325    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168353 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168359    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168600 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168380    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168600 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168397    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168600 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168600 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168437    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168600 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168462    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntzrl\" (UniqueName: \"kubernetes.io/projected/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-kube-api-access-ntzrl\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168866 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168659    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168866 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168704    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.168866 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.168734    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169340 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169183    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169553    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169600    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169627    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169653    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169678    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169712    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169763    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169795    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169800    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.169882 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169862    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.170421 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169892    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.170421 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169929    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-config\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.171406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.171378    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.171497 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.171409    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.172087 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.171721    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.172087 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.169625    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.172087 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.171807    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.172326 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.172271    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.172492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.172442    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.173018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.172853    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.173018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.172958    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.173018 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.172981    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-config\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.173252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.173168    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.173645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.173617    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.173887 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.173866    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.174331 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.174310    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.174461 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.174444    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.174655 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.174640    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.176973 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.176953    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntzrl\" (UniqueName: \"kubernetes.io/projected/8f4c7246-27bc-4c0d-83e3-488ce3e03d71-kube-api-access-ntzrl\") pod \"prometheus-k8s-0\" (UID: \"8f4c7246-27bc-4c0d-83e3-488ce3e03d71\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.368366 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.368332    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:08:06.519583 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.519547    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 02 19:08:06.522227 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:08:06.522200    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4c7246_27bc_4c0d_83e3_488ce3e03d71.slice/crio-e3497031750d669603d3924ca1171f462d2eef36837f6af6a10dfd360eaf6573 WatchSource:0}: Error finding container e3497031750d669603d3924ca1171f462d2eef36837f6af6a10dfd360eaf6573: Status 404 returned error can't find the container with id e3497031750d669603d3924ca1171f462d2eef36837f6af6a10dfd360eaf6573
Jul 02 19:08:06.990627 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.990594    2585 generic.go:356] "Generic (PLEG): container finished" podID="8f4c7246-27bc-4c0d-83e3-488ce3e03d71" containerID="61e75c26761ffb3bf8429997006132047edec0f6a9bb32affa005dd1a989c01d" exitCode=0
Jul 02 19:08:06.990823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.990675    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerDied","Data":"61e75c26761ffb3bf8429997006132047edec0f6a9bb32affa005dd1a989c01d"}
Jul 02 19:08:06.990823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:06.990742    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"e3497031750d669603d3924ca1171f462d2eef36837f6af6a10dfd360eaf6573"}
Jul 02 19:08:07.314138 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.314108    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d5a3b63-770a-4dbe-a0a5-8799dc95e30e" path="/var/lib/kubelet/pods/7d5a3b63-770a-4dbe-a0a5-8799dc95e30e/volumes"
Jul 02 19:08:07.998616 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.998584    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"5d7744ca1c2e447cc3064cbdb12c795bebfa9359025b0dc5808ea64f73bf4b6b"}
Jul 02 19:08:07.998971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.998648    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"c996128a9f6621331cc5c536fa66bc1b89664df99e638ad34be78c522cb3c28d"}
Jul 02 19:08:07.998971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.998674    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"da1d4f17f97e8583e6f642d78e0f2f6f434a1ff06663d92790af8f55eac1fde9"}
Jul 02 19:08:07.998971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.998697    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"419d826724e6714499b041d5e87c39cf475daa1d35c1801a5830f8aa72c413a2"}
Jul 02 19:08:07.998971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.998728    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"a7d612b8de43c8748920d0c811243b1013b4e313784b68aeafb28541b790686e"}
Jul 02 19:08:07.998971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:07.998753    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f4c7246-27bc-4c0d-83e3-488ce3e03d71","Type":"ContainerStarted","Data":"5009e4b6fd85366c1ae4ec8ce4d194fa2482cce0e14fcac805811ca117d7971f"}
Jul 02 19:08:08.044648 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:08.044589    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.025672992 podStartE2EDuration="2.044572368s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.899376ms" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:08.043506256 +0000 UTC m=+169.193538240" watchObservedRunningTime="2026-07-02 19:08:08.044572368 +0000 UTC m=+169.194604351"
Jul 02 19:08:11.368984 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:08:11.368898    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:09:06.369001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:09:06.368966    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:09:06.384242 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:09:06.384217    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:09:07.206862 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:09:07.206835    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 02 19:10:19.142049 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:10:19.142019    2585 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 19:11:45.714271 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.714232    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"]
Jul 02 19:11:45.717488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.717472    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.720536 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.720514    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 02 19:11:45.720677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.720628    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 02 19:11:45.720677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.720632    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 02 19:11:45.720803 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.720693    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 02 19:11:45.722084 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.722066    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nmldj\""
Jul 02 19:11:45.732600 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.732577    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"]
Jul 02 19:11:45.890841 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.890806    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5nfn\" (UniqueName: \"kubernetes.io/projected/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-kube-api-access-l5nfn\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.890841 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.890844    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-tmp\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.891067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.890869    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.891067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.890940    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.891067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.890984    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-manifests\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.991730 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.991635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-manifests\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.991730 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.991695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5nfn\" (UniqueName: \"kubernetes.io/projected/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-kube-api-access-l5nfn\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.991730 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.991728    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-tmp\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.992002 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.991761    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.992002 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.991809    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.992221 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.992200    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-manifests\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.994027 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.994005    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-tmp\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.994271 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.994251    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:45.994449 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:45.994432    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:46.017184 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:46.017141    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5nfn\" (UniqueName: \"kubernetes.io/projected/e8d14bbf-3425-4043-82cf-3f78e17fb1ef-kube-api-access-l5nfn\") pod \"opendatahub-operator-controller-manager-6cf784d988-clkfx\" (UID: \"e8d14bbf-3425-4043-82cf-3f78e17fb1ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:46.028022 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:46.027993    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:46.172691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:46.172660    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"]
Jul 02 19:11:46.176109 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:11:46.176079    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d14bbf_3425_4043_82cf_3f78e17fb1ef.slice/crio-4576d861e44b166c00a41bf81c9cc2c7359d56c48236fa9441fa785099f898e3 WatchSource:0}: Error finding container 4576d861e44b166c00a41bf81c9cc2c7359d56c48236fa9441fa785099f898e3: Status 404 returned error can't find the container with id 4576d861e44b166c00a41bf81c9cc2c7359d56c48236fa9441fa785099f898e3
Jul 02 19:11:46.177804 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:46.177788    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:11:46.718423 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:46.718389    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx" event={"ID":"e8d14bbf-3425-4043-82cf-3f78e17fb1ef","Type":"ContainerStarted","Data":"4576d861e44b166c00a41bf81c9cc2c7359d56c48236fa9441fa785099f898e3"}
Jul 02 19:11:49.733877 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:49.733832    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx" event={"ID":"e8d14bbf-3425-4043-82cf-3f78e17fb1ef","Type":"ContainerStarted","Data":"308e81a8b75d727432ad3d2fe65664e97fc1452dc852ab8fb9da023709de8c51"}
Jul 02 19:11:50.738571 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:50.738490    2585 generic.go:356] "Generic (PLEG): container finished" podID="e8d14bbf-3425-4043-82cf-3f78e17fb1ef" containerID="308e81a8b75d727432ad3d2fe65664e97fc1452dc852ab8fb9da023709de8c51" exitCode=0
Jul 02 19:11:50.738571 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:50.738528    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx" event={"ID":"e8d14bbf-3425-4043-82cf-3f78e17fb1ef","Type":"ContainerDied","Data":"308e81a8b75d727432ad3d2fe65664e97fc1452dc852ab8fb9da023709de8c51"}
Jul 02 19:11:51.743778 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:51.743743    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx" event={"ID":"e8d14bbf-3425-4043-82cf-3f78e17fb1ef","Type":"ContainerStarted","Data":"f61171fc777a240f3367e2d339ae6a2873350873d8acee3c1910c93e7b8d99a8"}
Jul 02 19:11:51.744196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:51.743872    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:11:51.769948 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:11:51.769889    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx" podStartSLOduration=2.198579708 podStartE2EDuration="6.769873114s" totalImagesPullingTime="3.472058789s" totalInitContainerRuntime="1.099234617s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:51.769375388 +0000 UTC m=+392.919407389" watchObservedRunningTime="2026-07-02 19:11:51.769873114 +0000 UTC m=+392.919905097"
Jul 02 19:12:02.748793 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:02.748761    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-clkfx"
Jul 02 19:12:06.446754 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.446712    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h4f6b"]
Jul 02 19:12:06.460332 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.460305    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:06.464841 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.464814    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 02 19:12:06.464841 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.464817    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nc4w5\""
Jul 02 19:12:06.483807 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.483774    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h4f6b"]
Jul 02 19:12:06.561973 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.561938    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p79m\" (UniqueName: \"kubernetes.io/projected/7a672aa1-157f-4759-831a-5467b13da7a4-kube-api-access-8p79m\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:06.561973 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.561978    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a672aa1-157f-4759-831a-5467b13da7a4-cert\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:06.662907 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.662872    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p79m\" (UniqueName: \"kubernetes.io/projected/7a672aa1-157f-4759-831a-5467b13da7a4-kube-api-access-8p79m\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:06.662907 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.662909    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a672aa1-157f-4759-831a-5467b13da7a4-cert\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:06.663108 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:06.663035    2585 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 02 19:12:06.663108 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:06.663106    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a672aa1-157f-4759-831a-5467b13da7a4-cert podName:7a672aa1-157f-4759-831a-5467b13da7a4 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:07.163073137 +0000 UTC m=+408.313105098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a672aa1-157f-4759-831a-5467b13da7a4-cert") pod "odh-model-controller-7998c67dc4-h4f6b" (UID: "7a672aa1-157f-4759-831a-5467b13da7a4") : secret "odh-model-controller-webhook-cert" not found
Jul 02 19:12:06.682611 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:06.682575    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p79m\" (UniqueName: \"kubernetes.io/projected/7a672aa1-157f-4759-831a-5467b13da7a4-kube-api-access-8p79m\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:07.167890 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.167856    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a672aa1-157f-4759-831a-5467b13da7a4-cert\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:07.170200 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.170169    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a672aa1-157f-4759-831a-5467b13da7a4-cert\") pod \"odh-model-controller-7998c67dc4-h4f6b\" (UID: \"7a672aa1-157f-4759-831a-5467b13da7a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:07.369609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.369575    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:07.536092 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.536049    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h4f6b"]
Jul 02 19:12:07.538612 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:07.538577    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a672aa1_157f_4759_831a_5467b13da7a4.slice/crio-6ca50aed6332e62ca4f824da1da93dec59a6c0cf5d0406eba9025411cb9ed3b4 WatchSource:0}: Error finding container 6ca50aed6332e62ca4f824da1da93dec59a6c0cf5d0406eba9025411cb9ed3b4: Status 404 returned error can't find the container with id 6ca50aed6332e62ca4f824da1da93dec59a6c0cf5d0406eba9025411cb9ed3b4
Jul 02 19:12:07.689889 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.689808    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sb69l"]
Jul 02 19:12:07.694262 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.694246    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:07.697194 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.697173    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 02 19:12:07.733233 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.733198    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sb69l"]
Jul 02 19:12:07.772728 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.772692    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4f541b1-420f-457c-9d27-89062ab4ddcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:07.772728 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.772729    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv6hb\" (UniqueName: \"kubernetes.io/projected/f4f541b1-420f-457c-9d27-89062ab4ddcc-kube-api-access-wv6hb\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:07.798540 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.798507    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b" event={"ID":"7a672aa1-157f-4759-831a-5467b13da7a4","Type":"ContainerStarted","Data":"6ca50aed6332e62ca4f824da1da93dec59a6c0cf5d0406eba9025411cb9ed3b4"}
Jul 02 19:12:07.874122 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.874086    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4f541b1-420f-457c-9d27-89062ab4ddcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:07.874122 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.874124    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv6hb\" (UniqueName: \"kubernetes.io/projected/f4f541b1-420f-457c-9d27-89062ab4ddcc-kube-api-access-wv6hb\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:07.874349 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:07.874278    2585 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 02 19:12:07.874349 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:07.874342    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f541b1-420f-457c-9d27-89062ab4ddcc-tls-certs podName:f4f541b1-420f-457c-9d27-89062ab4ddcc nodeName:}" failed. No retries permitted until 2026-07-02 19:12:08.374325687 +0000 UTC m=+409.524357648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f4f541b1-420f-457c-9d27-89062ab4ddcc-tls-certs") pod "model-serving-api-54d89bd5fc-sb69l" (UID: "f4f541b1-420f-457c-9d27-89062ab4ddcc") : secret "model-serving-api-tls" not found
Jul 02 19:12:07.902606 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:07.902577    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv6hb\" (UniqueName: \"kubernetes.io/projected/f4f541b1-420f-457c-9d27-89062ab4ddcc-kube-api-access-wv6hb\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:08.378729 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:08.378683    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4f541b1-420f-457c-9d27-89062ab4ddcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:08.381626 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:08.381595    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4f541b1-420f-457c-9d27-89062ab4ddcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-sb69l\" (UID: \"f4f541b1-420f-457c-9d27-89062ab4ddcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:08.604122 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:08.604085    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:08.786426 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:08.786392    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sb69l"]
Jul 02 19:12:08.793186 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:08.793141    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f541b1_420f_457c_9d27_89062ab4ddcc.slice/crio-86bf4a42543bb3329f3f7ee911e03ffd1b504ddd7481fcc422c714948af81b61 WatchSource:0}: Error finding container 86bf4a42543bb3329f3f7ee911e03ffd1b504ddd7481fcc422c714948af81b61: Status 404 returned error can't find the container with id 86bf4a42543bb3329f3f7ee911e03ffd1b504ddd7481fcc422c714948af81b61
Jul 02 19:12:08.803411 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:08.803379    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l" event={"ID":"f4f541b1-420f-457c-9d27-89062ab4ddcc","Type":"ContainerStarted","Data":"86bf4a42543bb3329f3f7ee911e03ffd1b504ddd7481fcc422c714948af81b61"}
Jul 02 19:12:09.393614 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.393575    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"]
Jul 02 19:12:09.399251 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.399212    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.406164 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.406124    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 02 19:12:09.406321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.406128    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 02 19:12:09.406321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.406128    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 02 19:12:09.406743 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.406713    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 02 19:12:09.406847 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.406747    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cb7hb\""
Jul 02 19:12:09.413541 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.413501    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"]
Jul 02 19:12:09.488799 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.488760    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa395899-6874-4a10-8c6b-397e8b0f9b29-tls-certs\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.488987 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.488820    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxv2n\" (UniqueName: \"kubernetes.io/projected/aa395899-6874-4a10-8c6b-397e8b0f9b29-kube-api-access-lxv2n\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.488987 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.488875    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa395899-6874-4a10-8c6b-397e8b0f9b29-tmp\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.589330 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.589294    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa395899-6874-4a10-8c6b-397e8b0f9b29-tmp\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.589508 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.589467    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa395899-6874-4a10-8c6b-397e8b0f9b29-tls-certs\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.589508 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.589498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxv2n\" (UniqueName: \"kubernetes.io/projected/aa395899-6874-4a10-8c6b-397e8b0f9b29-kube-api-access-lxv2n\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.589674 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:09.589652    2585 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 02 19:12:09.589738 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:09.589727    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa395899-6874-4a10-8c6b-397e8b0f9b29-tls-certs podName:aa395899-6874-4a10-8c6b-397e8b0f9b29 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:10.089704916 +0000 UTC m=+411.239736882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/aa395899-6874-4a10-8c6b-397e8b0f9b29-tls-certs") pod "kube-auth-proxy-c665485c8-w8wqk" (UID: "aa395899-6874-4a10-8c6b-397e8b0f9b29") : secret "kube-auth-proxy-tls" not found
Jul 02 19:12:09.591925 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.591878    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa395899-6874-4a10-8c6b-397e8b0f9b29-tmp\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:09.613613 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:09.613583    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxv2n\" (UniqueName: \"kubernetes.io/projected/aa395899-6874-4a10-8c6b-397e8b0f9b29-kube-api-access-lxv2n\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:10.093628 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.093582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa395899-6874-4a10-8c6b-397e8b0f9b29-tls-certs\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:10.096575 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.096548    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa395899-6874-4a10-8c6b-397e8b0f9b29-tls-certs\") pod \"kube-auth-proxy-c665485c8-w8wqk\" (UID: \"aa395899-6874-4a10-8c6b-397e8b0f9b29\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:10.311737 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.311702    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"
Jul 02 19:12:10.506485 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.506448    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c665485c8-w8wqk"]
Jul 02 19:12:10.509777 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:10.509743    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa395899_6874_4a10_8c6b_397e8b0f9b29.slice/crio-b480a9c57b6a7c4b1e7d0e79a969d340ad1b5d053cbee4f17dab7aa73f6d12d2 WatchSource:0}: Error finding container b480a9c57b6a7c4b1e7d0e79a969d340ad1b5d053cbee4f17dab7aa73f6d12d2: Status 404 returned error can't find the container with id b480a9c57b6a7c4b1e7d0e79a969d340ad1b5d053cbee4f17dab7aa73f6d12d2
Jul 02 19:12:10.811275 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.811236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk" event={"ID":"aa395899-6874-4a10-8c6b-397e8b0f9b29","Type":"ContainerStarted","Data":"b480a9c57b6a7c4b1e7d0e79a969d340ad1b5d053cbee4f17dab7aa73f6d12d2"}
Jul 02 19:12:10.812644 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.812618    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b" event={"ID":"7a672aa1-157f-4759-831a-5467b13da7a4","Type":"ContainerStarted","Data":"ea94f1a0c876a6af6e92ddcd92499ce4d5f8cd4a7ec00d7e641c41c13bb209a4"}
Jul 02 19:12:10.812768 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.812752    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:10.875001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.874950    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b" podStartSLOduration=1.99688147 podStartE2EDuration="4.874931692s" totalImagesPullingTime="2.878050222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:10.874836662 +0000 UTC m=+412.024868646" watchObservedRunningTime="2026-07-02 19:12:10.874931692 +0000 UTC m=+412.024963675"
Jul 02 19:12:10.998105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:10.998068    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"]
Jul 02 19:12:11.001676 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.001652    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.005345 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.005315    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 02 19:12:11.006163 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.006083    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 02 19:12:11.006163 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.006114    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 02 19:12:11.006689 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.006666    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m92gf\""
Jul 02 19:12:11.006897 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.006880    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 02 19:12:11.023831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.023639    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"]
Jul 02 19:12:11.102584 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102481    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-cacerts\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.102584 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102546    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.102818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102612    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.102818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102642    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j9hb\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-kube-api-access-6j9hb\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.102818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102700    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.102818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102775    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f53de26a-483e-4904-8d9e-5d63515a6bc0-local-certs\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.103016 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.102833    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-token\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203458 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-cacerts\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203458 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203469    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203490    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203604    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j9hb\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-kube-api-access-6j9hb\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203779 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203759    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203820 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203801    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f53de26a-483e-4904-8d9e-5d63515a6bc0-local-certs\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.203858 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.203841    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-token\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.204058 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.204030    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.206001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.205973    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f53de26a-483e-4904-8d9e-5d63515a6bc0-local-certs\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.206103 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.205999    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.206103 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.206040    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.206206 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.206182    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-cacerts\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.217316 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.217284    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j9hb\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-kube-api-access-6j9hb\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.217749 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.217729    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-token\") pod \"istiod-openshift-gateway-7574459656-lq9gl\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.313879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.313843    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:11.475227 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.475195    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"]
Jul 02 19:12:11.477917 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:11.477889    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53de26a_483e_4904_8d9e_5d63515a6bc0.slice/crio-24de562097f9d75292f40985e4667d2a375766e7f4d58dd767a92283020a91fe WatchSource:0}: Error finding container 24de562097f9d75292f40985e4667d2a375766e7f4d58dd767a92283020a91fe: Status 404 returned error can't find the container with id 24de562097f9d75292f40985e4667d2a375766e7f4d58dd767a92283020a91fe
Jul 02 19:12:11.818681 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:11.818640    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" event={"ID":"f53de26a-483e-4904-8d9e-5d63515a6bc0","Type":"ContainerStarted","Data":"24de562097f9d75292f40985e4667d2a375766e7f4d58dd767a92283020a91fe"}
Jul 02 19:12:12.826866 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:12.826831    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l" event={"ID":"f4f541b1-420f-457c-9d27-89062ab4ddcc","Type":"ContainerStarted","Data":"6622d2f824cc288a4708504e6ad4637766c5173aa580dafb975e5efaf14767e2"}
Jul 02 19:12:12.827438 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:12.827351    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:12.849367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:12.849304    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l" podStartSLOduration=2.498493493 podStartE2EDuration="5.849287466s" totalImagesPullingTime="3.350793973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:12.848163519 +0000 UTC m=+413.998195502" watchObservedRunningTime="2026-07-02 19:12:12.849287466 +0000 UTC m=+413.999319449"
Jul 02 19:12:14.527220 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.527168    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:12:14.527520 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.527261    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:12:14.837356 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.837258    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk" event={"ID":"aa395899-6874-4a10-8c6b-397e8b0f9b29","Type":"ContainerStarted","Data":"dc0a456a6b0b135e8d17c278a23de42ad9c0fcf83ddbb99578c26f25a281b3e3"}
Jul 02 19:12:14.838927 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.838899    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" event={"ID":"f53de26a-483e-4904-8d9e-5d63515a6bc0","Type":"ContainerStarted","Data":"165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570"}
Jul 02 19:12:14.839063 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.838964    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:14.878856 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.878797    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c665485c8-w8wqk" podStartSLOduration=1.915462886 podStartE2EDuration="5.878780314s" totalImagesPullingTime="3.963317428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:14.874719265 +0000 UTC m=+416.024751250" watchObservedRunningTime="2026-07-02 19:12:14.878780314 +0000 UTC m=+416.028812296"
Jul 02 19:12:14.928465 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:14.928399    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" podStartSLOduration=1.882062438 podStartE2EDuration="4.928378525s" totalImagesPullingTime="3.046316087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:14.927266533 +0000 UTC m=+416.077298534" watchObservedRunningTime="2026-07-02 19:12:14.928378525 +0000 UTC m=+416.078410508"
Jul 02 19:12:15.844557 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:15.844524    2585 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-lq9gl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 02 19:12:15.844924 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:15.844590    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" podUID="f53de26a-483e-4904-8d9e-5d63515a6bc0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 19:12:16.373941 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.373912    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"]
Jul 02 19:12:16.376782 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.376761    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.380653 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.380631    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-r2jl8\""
Jul 02 19:12:16.403071 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.403031    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"]
Jul 02 19:12:16.451733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451696    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.451733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451739    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.451971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451761    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.451971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451851    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.451971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451898    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.451971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451948    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.451971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.451968    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.452226 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.452038    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6abec698-bdf0-445a-ae12-2dd287f7e97b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.452226 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.452080    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4hvf\" (UniqueName: \"kubernetes.io/projected/6abec698-bdf0-445a-ae12-2dd287f7e97b-kube-api-access-s4hvf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553387 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553354    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553387 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553391    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553422    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6abec698-bdf0-445a-ae12-2dd287f7e97b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553472    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4hvf\" (UniqueName: \"kubernetes.io/projected/6abec698-bdf0-445a-ae12-2dd287f7e97b-kube-api-access-s4hvf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553536    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553558    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553581    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553609 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553605    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553860 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553682    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553868    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.553975 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.553955    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.554208 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.554187    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.554450 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.554427    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.554527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.554426    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6abec698-bdf0-445a-ae12-2dd287f7e97b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.556045 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.556027    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.556324 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.556304    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.576457 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.576422    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4hvf\" (UniqueName: \"kubernetes.io/projected/6abec698-bdf0-445a-ae12-2dd287f7e97b-kube-api-access-s4hvf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.580105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.580086    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6abec698-bdf0-445a-ae12-2dd287f7e97b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xxk8n\" (UID: \"6abec698-bdf0-445a-ae12-2dd287f7e97b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.687033 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.686998    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:16.861674 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:16.861641    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"]
Jul 02 19:12:16.864623 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:16.864596    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abec698_bdf0_445a_ae12_2dd287f7e97b.slice/crio-93319c1590ba492eaf0efa34873f25eafa4792441b8528dbae8f3bfb437c6228 WatchSource:0}: Error finding container 93319c1590ba492eaf0efa34873f25eafa4792441b8528dbae8f3bfb437c6228: Status 404 returned error can't find the container with id 93319c1590ba492eaf0efa34873f25eafa4792441b8528dbae8f3bfb437c6228
Jul 02 19:12:17.734220 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.734181    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"]
Jul 02 19:12:17.737808 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.737779    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:17.738746 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.738719    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"]
Jul 02 19:12:17.741793 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.741775    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:17.743790 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.743565    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 02 19:12:17.743790 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.743586    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tdwh8\""
Jul 02 19:12:17.746097 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.746074    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7h4cr\""
Jul 02 19:12:17.746911 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.746885    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 02 19:12:17.761752 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.761726    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"]
Jul 02 19:12:17.785583 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.785544    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"]
Jul 02 19:12:17.854374 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.854340    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n" event={"ID":"6abec698-bdf0-445a-ae12-2dd287f7e97b","Type":"ContainerStarted","Data":"93319c1590ba492eaf0efa34873f25eafa4792441b8528dbae8f3bfb437c6228"}
Jul 02 19:12:17.865844 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.865798    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrh9m\" (UniqueName: \"kubernetes.io/projected/3316bb71-40d7-45c4-a4de-d15de892d3d4-kube-api-access-vrh9m\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:17.865844 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.865845    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3316bb71-40d7-45c4-a4de-d15de892d3d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:17.866346 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.865929    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw62c\" (UniqueName: \"kubernetes.io/projected/5c88087d-ec69-42b3-bcef-05fd3f8683d5-kube-api-access-xw62c\") pod \"kserve-controller-manager-d6f4b65b8-kp7dg\" (UID: \"5c88087d-ec69-42b3-bcef-05fd3f8683d5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:17.866346 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.865989    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c88087d-ec69-42b3-bcef-05fd3f8683d5-cert\") pod \"kserve-controller-manager-d6f4b65b8-kp7dg\" (UID: \"5c88087d-ec69-42b3-bcef-05fd3f8683d5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:17.966625 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.966582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c88087d-ec69-42b3-bcef-05fd3f8683d5-cert\") pod \"kserve-controller-manager-d6f4b65b8-kp7dg\" (UID: \"5c88087d-ec69-42b3-bcef-05fd3f8683d5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:17.966813 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.966678    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrh9m\" (UniqueName: \"kubernetes.io/projected/3316bb71-40d7-45c4-a4de-d15de892d3d4-kube-api-access-vrh9m\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:17.966813 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.966718    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3316bb71-40d7-45c4-a4de-d15de892d3d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:17.966813 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.966759    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw62c\" (UniqueName: \"kubernetes.io/projected/5c88087d-ec69-42b3-bcef-05fd3f8683d5-kube-api-access-xw62c\") pod \"kserve-controller-manager-d6f4b65b8-kp7dg\" (UID: \"5c88087d-ec69-42b3-bcef-05fd3f8683d5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:17.967031 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:17.966851    2585 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 02 19:12:17.967031 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:17.966926    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3316bb71-40d7-45c4-a4de-d15de892d3d4-cert podName:3316bb71-40d7-45c4-a4de-d15de892d3d4 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:18.466905141 +0000 UTC m=+419.616937120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3316bb71-40d7-45c4-a4de-d15de892d3d4-cert") pod "llmisvc-controller-manager-777c6bd7d5-m57zn" (UID: "3316bb71-40d7-45c4-a4de-d15de892d3d4") : secret "llmisvc-webhook-server-cert" not found
Jul 02 19:12:17.969450 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.969421    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c88087d-ec69-42b3-bcef-05fd3f8683d5-cert\") pod \"kserve-controller-manager-d6f4b65b8-kp7dg\" (UID: \"5c88087d-ec69-42b3-bcef-05fd3f8683d5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:17.978032 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.978007    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrh9m\" (UniqueName: \"kubernetes.io/projected/3316bb71-40d7-45c4-a4de-d15de892d3d4-kube-api-access-vrh9m\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:17.978328 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:17.978295    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw62c\" (UniqueName: \"kubernetes.io/projected/5c88087d-ec69-42b3-bcef-05fd3f8683d5-kube-api-access-xw62c\") pod \"kserve-controller-manager-d6f4b65b8-kp7dg\" (UID: \"5c88087d-ec69-42b3-bcef-05fd3f8683d5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:18.053458 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.053364    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:18.221260 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.221235    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"]
Jul 02 19:12:18.223685 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:18.223657    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c88087d_ec69_42b3_bcef_05fd3f8683d5.slice/crio-aa5156cf638d9d7197bda577280fe1d6aa87633998e6a809087efd8b44ce032d WatchSource:0}: Error finding container aa5156cf638d9d7197bda577280fe1d6aa87633998e6a809087efd8b44ce032d: Status 404 returned error can't find the container with id aa5156cf638d9d7197bda577280fe1d6aa87633998e6a809087efd8b44ce032d
Jul 02 19:12:18.472559 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.472523    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3316bb71-40d7-45c4-a4de-d15de892d3d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:18.475308 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.475274    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3316bb71-40d7-45c4-a4de-d15de892d3d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m57zn\" (UID: \"3316bb71-40d7-45c4-a4de-d15de892d3d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:18.660491 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.660456    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:18.844376 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.844292    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:18.861660 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:18.861618    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg" event={"ID":"5c88087d-ec69-42b3-bcef-05fd3f8683d5","Type":"ContainerStarted","Data":"aa5156cf638d9d7197bda577280fe1d6aa87633998e6a809087efd8b44ce032d"}
Jul 02 19:12:19.743989 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:19.743953    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:12:19.744367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:19.744030    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:12:19.744367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:19.744064    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:12:19.828243 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:19.828212    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"]
Jul 02 19:12:19.870020 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:19.869972    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n" event={"ID":"6abec698-bdf0-445a-ae12-2dd287f7e97b","Type":"ContainerStarted","Data":"ea0a828e3bd1a7691aec57f2828aac9d985ed95e4433c7d82ac2bda1cd319add"}
Jul 02 19:12:19.892258 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:19.892188    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n" podStartSLOduration=1.015379061 podStartE2EDuration="3.892136938s" totalImagesPullingTime="2.876757877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:19.890020514 +0000 UTC m=+421.040052516" watchObservedRunningTime="2026-07-02 19:12:19.892136938 +0000 UTC m=+421.042168922"
Jul 02 19:12:20.358345 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:12:20.358301    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3316bb71_40d7_45c4_a4de_d15de892d3d4.slice/crio-5b5a57d667138172e0665572c7385bbdef21c727139ff5ba688afd52eeb9284b WatchSource:0}: Error finding container 5b5a57d667138172e0665572c7385bbdef21c727139ff5ba688afd52eeb9284b: Status 404 returned error can't find the container with id 5b5a57d667138172e0665572c7385bbdef21c727139ff5ba688afd52eeb9284b
Jul 02 19:12:20.606570 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.606539    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"]
Jul 02 19:12:20.606730 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.606711    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" podUID="f53de26a-483e-4904-8d9e-5d63515a6bc0" containerName="discovery" containerID="cri-o://165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570" gracePeriod=30
Jul 02 19:12:20.687287 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.687255    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:20.691949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.691923    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:20.862111 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.859426    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:20.876250 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.876161    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn" event={"ID":"3316bb71-40d7-45c4-a4de-d15de892d3d4","Type":"ContainerStarted","Data":"5b5a57d667138172e0665572c7385bbdef21c727139ff5ba688afd52eeb9284b"}
Jul 02 19:12:20.877760 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.877734    2585 generic.go:356] "Generic (PLEG): container finished" podID="f53de26a-483e-4904-8d9e-5d63515a6bc0" containerID="165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570" exitCode=0
Jul 02 19:12:20.877896 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.877779    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"
Jul 02 19:12:20.877896 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.877822    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" event={"ID":"f53de26a-483e-4904-8d9e-5d63515a6bc0","Type":"ContainerDied","Data":"165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570"}
Jul 02 19:12:20.877983 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.877897    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl" event={"ID":"f53de26a-483e-4904-8d9e-5d63515a6bc0","Type":"ContainerDied","Data":"24de562097f9d75292f40985e4667d2a375766e7f4d58dd767a92283020a91fe"}
Jul 02 19:12:20.878449 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.877931    2585 scope.go:117] "RemoveContainer" containerID="165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570"
Jul 02 19:12:20.879603 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.879505    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg" event={"ID":"5c88087d-ec69-42b3-bcef-05fd3f8683d5","Type":"ContainerStarted","Data":"6fd0e972008b44593f029cc7b7962ec22ceb764667249581803879395cd859c4"}
Jul 02 19:12:20.879740 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.879722    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:20.880840 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.880822    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xxk8n"
Jul 02 19:12:20.905427 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.905377    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg" podStartSLOduration=1.718836223 podStartE2EDuration="3.90536013s" totalImagesPullingTime="2.186523907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:20.90382096 +0000 UTC m=+422.053852942" watchObservedRunningTime="2026-07-02 19:12:20.90536013 +0000 UTC m=+422.055392112"
Jul 02 19:12:20.929053 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.929027    2585 scope.go:117] "RemoveContainer" containerID="165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570"
Jul 02 19:12:20.929389 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:12:20.929367    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570\": container with ID starting with 165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570 not found: ID does not exist" containerID="165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570"
Jul 02 19:12:20.929483 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.929398    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570"} err="failed to get container status \"165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570\": rpc error: code = NotFound desc = could not find container \"165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570\": container with ID starting with 165bb4356d2a9f86606b2f589c8ab973547fd0be895f586b02bbca9428561570 not found: ID does not exist"
Jul 02 19:12:20.995286 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995256    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-ca-configmap\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995465 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995295    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-token\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995465 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995322    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-kubeconfig\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995465 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995346    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-dns-cert\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995465 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995386    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j9hb\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-kube-api-access-6j9hb\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995465 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995436    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f53de26a-483e-4904-8d9e-5d63515a6bc0-local-certs\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995731 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995469    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-cacerts\") pod \"f53de26a-483e-4904-8d9e-5d63515a6bc0\" (UID: \"f53de26a-483e-4904-8d9e-5d63515a6bc0\") "
Jul 02 19:12:20.995806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995722    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:12:20.996490 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.995917    2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-ca-configmap\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:20.997930 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.997889    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-kube-api-access-6j9hb" (OuterVolumeSpecName: "kube-api-access-6j9hb") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "kube-api-access-6j9hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:12:20.998041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.997985    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:12:20.998115 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.998024    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:12:20.998392 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.998370    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:12:20.998477 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.998395    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-cacerts" (OuterVolumeSpecName: "cacerts") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:12:20.998477 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:20.998389    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53de26a-483e-4904-8d9e-5d63515a6bc0-local-certs" (OuterVolumeSpecName: "local-certs") pod "f53de26a-483e-4904-8d9e-5d63515a6bc0" (UID: "f53de26a-483e-4904-8d9e-5d63515a6bc0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:12:21.096823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.096790    2585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-cacerts\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:21.096823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.096820    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-token\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:21.096823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.096831    2585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-kubeconfig\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:21.097196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.096840    2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f53de26a-483e-4904-8d9e-5d63515a6bc0-istio-csr-dns-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:21.097196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.096850    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j9hb\" (UniqueName: \"kubernetes.io/projected/f53de26a-483e-4904-8d9e-5d63515a6bc0-kube-api-access-6j9hb\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:21.097196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.096858    2585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f53de26a-483e-4904-8d9e-5d63515a6bc0-local-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:12:21.200649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.200617    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"]
Jul 02 19:12:21.204675 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.204648    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lq9gl"]
Jul 02 19:12:21.316700 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.316666    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f53de26a-483e-4904-8d9e-5d63515a6bc0" path="/var/lib/kubelet/pods/f53de26a-483e-4904-8d9e-5d63515a6bc0/volumes"
Jul 02 19:12:21.821175 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.821135    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-h4f6b"
Jul 02 19:12:21.886687 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:21.886653    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:23.836715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:23.836682    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-sb69l"
Jul 02 19:12:23.897657 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:23.897622    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn" event={"ID":"3316bb71-40d7-45c4-a4de-d15de892d3d4","Type":"ContainerStarted","Data":"8779318858e02e981b5996dd1a8f1b6acae45c31e410e348fe10e676f66f73c2"}
Jul 02 19:12:23.897842 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:23.897720    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:12:23.914514 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:23.914469    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn" podStartSLOduration=4.174773966 podStartE2EDuration="6.914452468s" totalImagesPullingTime="2.739678502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:23.913654224 +0000 UTC m=+425.063686208" watchObservedRunningTime="2026-07-02 19:12:23.914452468 +0000 UTC m=+425.064484493"
Jul 02 19:12:52.897601 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:52.897531    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kp7dg"
Jul 02 19:12:54.904640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:12:54.904610    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m57zn"
Jul 02 19:13:07.139584 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.139545    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k6gzt"]
Jul 02 19:13:07.139960 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.139892    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53de26a-483e-4904-8d9e-5d63515a6bc0" containerName="discovery"
Jul 02 19:13:07.139960 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.139902    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53de26a-483e-4904-8d9e-5d63515a6bc0" containerName="discovery"
Jul 02 19:13:07.140032 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.139967    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53de26a-483e-4904-8d9e-5d63515a6bc0" containerName="discovery"
Jul 02 19:13:07.155760 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.155725    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k6gzt"]
Jul 02 19:13:07.155932 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.155817    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:07.158709 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.158681    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 02 19:13:07.158843 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.158734    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 02 19:13:07.159059 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.159038    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2bnrm\""
Jul 02 19:13:07.189491 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.189469    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trxgm\" (UniqueName: \"kubernetes.io/projected/c02aa8d0-ec86-470c-b893-5d6ca4c21843-kube-api-access-trxgm\") pod \"kuadrant-operator-catalog-k6gzt\" (UID: \"c02aa8d0-ec86-470c-b893-5d6ca4c21843\") " pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:07.290160 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.290116    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trxgm\" (UniqueName: \"kubernetes.io/projected/c02aa8d0-ec86-470c-b893-5d6ca4c21843-kube-api-access-trxgm\") pod \"kuadrant-operator-catalog-k6gzt\" (UID: \"c02aa8d0-ec86-470c-b893-5d6ca4c21843\") " pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:07.298940 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.298911    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trxgm\" (UniqueName: \"kubernetes.io/projected/c02aa8d0-ec86-470c-b893-5d6ca4c21843-kube-api-access-trxgm\") pod \"kuadrant-operator-catalog-k6gzt\" (UID: \"c02aa8d0-ec86-470c-b893-5d6ca4c21843\") " pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:07.449167 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.449053    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k6gzt"]
Jul 02 19:13:07.449413 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.449396    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:07.573909 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.573884    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k6gzt"]
Jul 02 19:13:07.576396 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:07.576370    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02aa8d0_ec86_470c_b893_5d6ca4c21843.slice/crio-d326d087ce15eb81bf729dad646b640626bad033c8e06f128fc0299dcdcad307 WatchSource:0}: Error finding container d326d087ce15eb81bf729dad646b640626bad033c8e06f128fc0299dcdcad307: Status 404 returned error can't find the container with id d326d087ce15eb81bf729dad646b640626bad033c8e06f128fc0299dcdcad307
Jul 02 19:13:07.658707 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.658665    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tpjqf"]
Jul 02 19:13:07.663183 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.663167    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:07.670109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.670082    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tpjqf"]
Jul 02 19:13:07.693792 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.693757    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfnj5\" (UniqueName: \"kubernetes.io/projected/55e88b02-564b-4045-bca3-d8f9986a196b-kube-api-access-dfnj5\") pod \"kuadrant-operator-catalog-tpjqf\" (UID: \"55e88b02-564b-4045-bca3-d8f9986a196b\") " pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:07.795139 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.795050    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfnj5\" (UniqueName: \"kubernetes.io/projected/55e88b02-564b-4045-bca3-d8f9986a196b-kube-api-access-dfnj5\") pod \"kuadrant-operator-catalog-tpjqf\" (UID: \"55e88b02-564b-4045-bca3-d8f9986a196b\") " pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:07.803576 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.803552    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfnj5\" (UniqueName: \"kubernetes.io/projected/55e88b02-564b-4045-bca3-d8f9986a196b-kube-api-access-dfnj5\") pod \"kuadrant-operator-catalog-tpjqf\" (UID: \"55e88b02-564b-4045-bca3-d8f9986a196b\") " pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:07.973495 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:07.973464    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:08.069178 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:08.069127    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt" event={"ID":"c02aa8d0-ec86-470c-b893-5d6ca4c21843","Type":"ContainerStarted","Data":"d326d087ce15eb81bf729dad646b640626bad033c8e06f128fc0299dcdcad307"}
Jul 02 19:13:08.117204 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:08.117168    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tpjqf"]
Jul 02 19:13:08.119979 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:08.119952    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e88b02_564b_4045_bca3_d8f9986a196b.slice/crio-15b72d01eafa107080f27f4fec597be50a519a3374d6ddd6f6d92cf7db0a506e WatchSource:0}: Error finding container 15b72d01eafa107080f27f4fec597be50a519a3374d6ddd6f6d92cf7db0a506e: Status 404 returned error can't find the container with id 15b72d01eafa107080f27f4fec597be50a519a3374d6ddd6f6d92cf7db0a506e
Jul 02 19:13:09.075510 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:09.075479    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf" event={"ID":"55e88b02-564b-4045-bca3-d8f9986a196b","Type":"ContainerStarted","Data":"15b72d01eafa107080f27f4fec597be50a519a3374d6ddd6f6d92cf7db0a506e"}
Jul 02 19:13:10.080427 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.080388    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt" event={"ID":"c02aa8d0-ec86-470c-b893-5d6ca4c21843","Type":"ContainerStarted","Data":"6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1"}
Jul 02 19:13:10.080858 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.080456    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt" podUID="c02aa8d0-ec86-470c-b893-5d6ca4c21843" containerName="registry-server" containerID="cri-o://6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1" gracePeriod=2
Jul 02 19:13:10.081852 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.081826    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf" event={"ID":"55e88b02-564b-4045-bca3-d8f9986a196b","Type":"ContainerStarted","Data":"4511637c780cfa347d2efb90f7222fea4c8938ce2fcb2c84b3d84b5aa5a872ca"}
Jul 02 19:13:10.097272 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.097222    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt" podStartSLOduration=1.091905679 podStartE2EDuration="3.097205675s" totalImagesPullingTime="2.005299996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:10.096416826 +0000 UTC m=+471.246448809" watchObservedRunningTime="2026-07-02 19:13:10.097205675 +0000 UTC m=+471.247237657"
Jul 02 19:13:10.112406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.112371    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf" podStartSLOduration=1.589252372 podStartE2EDuration="3.112358428s" totalImagesPullingTime="1.523106056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:10.111960432 +0000 UTC m=+471.261992416" watchObservedRunningTime="2026-07-02 19:13:10.112358428 +0000 UTC m=+471.262390411"
Jul 02 19:13:10.329008 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.328982    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:10.417078 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.417049    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trxgm\" (UniqueName: \"kubernetes.io/projected/c02aa8d0-ec86-470c-b893-5d6ca4c21843-kube-api-access-trxgm\") pod \"c02aa8d0-ec86-470c-b893-5d6ca4c21843\" (UID: \"c02aa8d0-ec86-470c-b893-5d6ca4c21843\") "
Jul 02 19:13:10.418995 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.418975    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02aa8d0-ec86-470c-b893-5d6ca4c21843-kube-api-access-trxgm" (OuterVolumeSpecName: "kube-api-access-trxgm") pod "c02aa8d0-ec86-470c-b893-5d6ca4c21843" (UID: "c02aa8d0-ec86-470c-b893-5d6ca4c21843"). InnerVolumeSpecName "kube-api-access-trxgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:13:10.518441 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:10.518400    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trxgm\" (UniqueName: \"kubernetes.io/projected/c02aa8d0-ec86-470c-b893-5d6ca4c21843-kube-api-access-trxgm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:13:11.086563 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.086531    2585 generic.go:356] "Generic (PLEG): container finished" podID="c02aa8d0-ec86-470c-b893-5d6ca4c21843" containerID="6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1" exitCode=0
Jul 02 19:13:11.086986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.086579    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt"
Jul 02 19:13:11.086986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.086627    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt" event={"ID":"c02aa8d0-ec86-470c-b893-5d6ca4c21843","Type":"ContainerDied","Data":"6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1"}
Jul 02 19:13:11.086986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.086709    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k6gzt" event={"ID":"c02aa8d0-ec86-470c-b893-5d6ca4c21843","Type":"ContainerDied","Data":"d326d087ce15eb81bf729dad646b640626bad033c8e06f128fc0299dcdcad307"}
Jul 02 19:13:11.086986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.086741    2585 scope.go:117] "RemoveContainer" containerID="6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1"
Jul 02 19:13:11.095623 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.095608    2585 scope.go:117] "RemoveContainer" containerID="6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1"
Jul 02 19:13:11.095883 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:11.095866    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1\": container with ID starting with 6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1 not found: ID does not exist" containerID="6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1"
Jul 02 19:13:11.095929 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.095891    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1"} err="failed to get container status \"6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1\": rpc error: code = NotFound desc = could not find container \"6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1\": container with ID starting with 6f816bd12233882ef733652ee7a8abffb06cf91c10c54062a174650667879ea1 not found: ID does not exist"
Jul 02 19:13:11.106665 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.106634    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k6gzt"]
Jul 02 19:13:11.108483 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.108450    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k6gzt"]
Jul 02 19:13:11.319159 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:11.319116    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02aa8d0-ec86-470c-b893-5d6ca4c21843" path="/var/lib/kubelet/pods/c02aa8d0-ec86-470c-b893-5d6ca4c21843/volumes"
Jul 02 19:13:17.974556 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:17.974514    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:17.975045 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:17.974604    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:17.996480 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:17.996456    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:18.138427 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:18.138403    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-tpjqf"
Jul 02 19:13:28.674466 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.674428    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"]
Jul 02 19:13:28.674934 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.674731    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c02aa8d0-ec86-470c-b893-5d6ca4c21843" containerName="registry-server"
Jul 02 19:13:28.674934 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.674741    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02aa8d0-ec86-470c-b893-5d6ca4c21843" containerName="registry-server"
Jul 02 19:13:28.674934 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.674795    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c02aa8d0-ec86-470c-b893-5d6ca4c21843" containerName="registry-server"
Jul 02 19:13:28.678946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.678927    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:28.682441 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.682421    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 02 19:13:28.682564 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.682507    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8dcg8\""
Jul 02 19:13:28.698089 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.697583    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"]
Jul 02 19:13:28.777631 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.777592    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pct2n\" (UniqueName: \"kubernetes.io/projected/8db751b7-4a83-4ba8-9f63-5c44535701cd-kube-api-access-pct2n\") pod \"dns-operator-controller-manager-864dd78c4c-j84cp\" (UID: \"8db751b7-4a83-4ba8-9f63-5c44535701cd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:28.878363 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.878328    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pct2n\" (UniqueName: \"kubernetes.io/projected/8db751b7-4a83-4ba8-9f63-5c44535701cd-kube-api-access-pct2n\") pod \"dns-operator-controller-manager-864dd78c4c-j84cp\" (UID: \"8db751b7-4a83-4ba8-9f63-5c44535701cd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:28.891144 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.891113    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pct2n\" (UniqueName: \"kubernetes.io/projected/8db751b7-4a83-4ba8-9f63-5c44535701cd-kube-api-access-pct2n\") pod \"dns-operator-controller-manager-864dd78c4c-j84cp\" (UID: \"8db751b7-4a83-4ba8-9f63-5c44535701cd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:28.988760 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:28.988671    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:29.176577 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:29.176552    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"]
Jul 02 19:13:29.177991 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:29.177959    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db751b7_4a83_4ba8_9f63_5c44535701cd.slice/crio-531fed763e048a76281992ccf109c3b035ea8431d5e56aba3ecbea5b915ed3e2 WatchSource:0}: Error finding container 531fed763e048a76281992ccf109c3b035ea8431d5e56aba3ecbea5b915ed3e2: Status 404 returned error can't find the container with id 531fed763e048a76281992ccf109c3b035ea8431d5e56aba3ecbea5b915ed3e2
Jul 02 19:13:30.174948 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:30.174905    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp" event={"ID":"8db751b7-4a83-4ba8-9f63-5c44535701cd","Type":"ContainerStarted","Data":"531fed763e048a76281992ccf109c3b035ea8431d5e56aba3ecbea5b915ed3e2"}
Jul 02 19:13:31.182345 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:31.182299    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp" event={"ID":"8db751b7-4a83-4ba8-9f63-5c44535701cd","Type":"ContainerStarted","Data":"5fc12a01fa837952bfa7d425f810b15936a80e8ced5fdebbae2b0e02af360a2f"}
Jul 02 19:13:31.182859 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:31.182834    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:31.205237 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:31.205192    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp" podStartSLOduration=1.379682241 podStartE2EDuration="3.205177567s" totalImagesPullingTime="1.825495326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:31.202173487 +0000 UTC m=+492.352205470" watchObservedRunningTime="2026-07-02 19:13:31.205177567 +0000 UTC m=+492.355209546"
Jul 02 19:13:36.243075 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.243039    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"]
Jul 02 19:13:36.246384 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.246369    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.249132 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.249108    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8jxtl\""
Jul 02 19:13:36.257330 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.257306    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"]
Jul 02 19:13:36.345361 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.345323    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ac78a43-7a43-416e-8ec1-ebdf29592f38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.345543 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.345375    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzsk6\" (UniqueName: \"kubernetes.io/projected/7ac78a43-7a43-416e-8ec1-ebdf29592f38-kube-api-access-gzsk6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.446861 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.446808    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ac78a43-7a43-416e-8ec1-ebdf29592f38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.447025 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.446960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzsk6\" (UniqueName: \"kubernetes.io/projected/7ac78a43-7a43-416e-8ec1-ebdf29592f38-kube-api-access-gzsk6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.447298 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.447277    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ac78a43-7a43-416e-8ec1-ebdf29592f38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.464203 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.464172    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzsk6\" (UniqueName: \"kubernetes.io/projected/7ac78a43-7a43-416e-8ec1-ebdf29592f38-kube-api-access-gzsk6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.556258 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.556169    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:36.689270 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:36.689241    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"]
Jul 02 19:13:36.693187 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:36.693130    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac78a43_7a43_416e_8ec1_ebdf29592f38.slice/crio-ebdbcebba78af4618ce3255058f51c7ca97b7ffa99ea34f9d49640bd7b67e0bf WatchSource:0}: Error finding container ebdbcebba78af4618ce3255058f51c7ca97b7ffa99ea34f9d49640bd7b67e0bf: Status 404 returned error can't find the container with id ebdbcebba78af4618ce3255058f51c7ca97b7ffa99ea34f9d49640bd7b67e0bf
Jul 02 19:13:37.208999 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:37.208955    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2" event={"ID":"7ac78a43-7a43-416e-8ec1-ebdf29592f38","Type":"ContainerStarted","Data":"ebdbcebba78af4618ce3255058f51c7ca97b7ffa99ea34f9d49640bd7b67e0bf"}
Jul 02 19:13:40.653493 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.653455    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"]
Jul 02 19:13:40.657375 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.657351    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:40.660275 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.660254    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bptpc\""
Jul 02 19:13:40.667438 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.667409    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"]
Jul 02 19:13:40.686157 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.686119    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgxwq\" (UniqueName: \"kubernetes.io/projected/32430e5d-8fa6-4a5e-8d9d-8008f2269dc9-kube-api-access-pgxwq\") pod \"authorino-operator-75cc8d5cb7-6t5g9\" (UID: \"32430e5d-8fa6-4a5e-8d9d-8008f2269dc9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:40.787062 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.787024    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgxwq\" (UniqueName: \"kubernetes.io/projected/32430e5d-8fa6-4a5e-8d9d-8008f2269dc9-kube-api-access-pgxwq\") pod \"authorino-operator-75cc8d5cb7-6t5g9\" (UID: \"32430e5d-8fa6-4a5e-8d9d-8008f2269dc9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:40.801613 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.801581    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgxwq\" (UniqueName: \"kubernetes.io/projected/32430e5d-8fa6-4a5e-8d9d-8008f2269dc9-kube-api-access-pgxwq\") pod \"authorino-operator-75cc8d5cb7-6t5g9\" (UID: \"32430e5d-8fa6-4a5e-8d9d-8008f2269dc9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:40.970552 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:40.970473    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:41.618996 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:41.618934    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"]
Jul 02 19:13:41.621473 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:41.621439    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32430e5d_8fa6_4a5e_8d9d_8008f2269dc9.slice/crio-916998243cdcc0994fc92c02aabc7f214655af50e9850bc95ce799ca71513b91 WatchSource:0}: Error finding container 916998243cdcc0994fc92c02aabc7f214655af50e9850bc95ce799ca71513b91: Status 404 returned error can't find the container with id 916998243cdcc0994fc92c02aabc7f214655af50e9850bc95ce799ca71513b91
Jul 02 19:13:42.189403 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:42.189371    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j84cp"
Jul 02 19:13:42.233736 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:42.233703    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2" event={"ID":"7ac78a43-7a43-416e-8ec1-ebdf29592f38","Type":"ContainerStarted","Data":"4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b"}
Jul 02 19:13:42.233906 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:42.233777    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:42.235073 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:42.235050    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9" event={"ID":"32430e5d-8fa6-4a5e-8d9d-8008f2269dc9","Type":"ContainerStarted","Data":"916998243cdcc0994fc92c02aabc7f214655af50e9850bc95ce799ca71513b91"}
Jul 02 19:13:42.271789 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:42.271708    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2" podStartSLOduration=1.409784682 podStartE2EDuration="6.2716894s" totalImagesPullingTime="4.861904718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:42.270440968 +0000 UTC m=+503.420472954" watchObservedRunningTime="2026-07-02 19:13:42.2716894 +0000 UTC m=+503.421721384"
Jul 02 19:13:44.245539 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.245505    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9" event={"ID":"32430e5d-8fa6-4a5e-8d9d-8008f2269dc9","Type":"ContainerStarted","Data":"9a673766842c363b5bfcb638cf1ca73293699a471fee828e0dc10545c577b849"}
Jul 02 19:13:44.245961 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.245649    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:44.268598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.268544    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9" podStartSLOduration=2.111326283 podStartE2EDuration="4.268527209s" totalImagesPullingTime="2.157200926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:44.266215105 +0000 UTC m=+505.416247088" watchObservedRunningTime="2026-07-02 19:13:44.268527209 +0000 UTC m=+505.418559554"
Jul 02 19:13:44.586052 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.585975    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"]
Jul 02 19:13:44.588836 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.588820    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.591648 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.591624    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 02 19:13:44.591759 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.591698    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2chrd\""
Jul 02 19:13:44.592180 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.592140    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 02 19:13:44.601766 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.601744    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"]
Jul 02 19:13:44.618627 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.618596    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4nh8\" (UniqueName: \"kubernetes.io/projected/6360c37a-f946-46e8-a1fd-6347e3064d4d-kube-api-access-t4nh8\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.618749 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.618655    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6360c37a-f946-46e8-a1fd-6347e3064d4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.618795 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.618762    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6360c37a-f946-46e8-a1fd-6347e3064d4d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.635707 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.635680    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"]
Jul 02 19:13:44.638884 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.638867    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:44.641550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.641531    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qvvqh\""
Jul 02 19:13:44.647785 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.647765    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"]
Jul 02 19:13:44.719819 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.719768    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znxkj\" (UniqueName: \"kubernetes.io/projected/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6-kube-api-access-znxkj\") pod \"limitador-operator-controller-manager-859d7b78bd-gqh28\" (UID: \"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:44.719975 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.719831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6360c37a-f946-46e8-a1fd-6347e3064d4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.719975 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.719889    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6360c37a-f946-46e8-a1fd-6347e3064d4d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.719975 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.719921    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4nh8\" (UniqueName: \"kubernetes.io/projected/6360c37a-f946-46e8-a1fd-6347e3064d4d-kube-api-access-t4nh8\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.719975 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:44.719955    2585 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 02 19:13:44.720130 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:44.720047    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6360c37a-f946-46e8-a1fd-6347e3064d4d-plugin-serving-cert podName:6360c37a-f946-46e8-a1fd-6347e3064d4d nodeName:}" failed. No retries permitted until 2026-07-02 19:13:45.220025855 +0000 UTC m=+506.370057833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6360c37a-f946-46e8-a1fd-6347e3064d4d-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-pdmlw" (UID: "6360c37a-f946-46e8-a1fd-6347e3064d4d") : secret "plugin-serving-cert" not found
Jul 02 19:13:44.720589 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.720572    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6360c37a-f946-46e8-a1fd-6347e3064d4d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.731222 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.731196    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4nh8\" (UniqueName: \"kubernetes.io/projected/6360c37a-f946-46e8-a1fd-6347e3064d4d-kube-api-access-t4nh8\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:44.820329 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.820298    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znxkj\" (UniqueName: \"kubernetes.io/projected/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6-kube-api-access-znxkj\") pod \"limitador-operator-controller-manager-859d7b78bd-gqh28\" (UID: \"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:44.829237 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.829211    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znxkj\" (UniqueName: \"kubernetes.io/projected/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6-kube-api-access-znxkj\") pod \"limitador-operator-controller-manager-859d7b78bd-gqh28\" (UID: \"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:44.950317 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:44.950288    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:45.085106 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:45.085083    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"]
Jul 02 19:13:45.087329 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:45.087296    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1d37bd_40fc_4c9e_8a8a_9a1a74d733f6.slice/crio-14fb6b7ba770140322c060966d55bd272d29e3d265dc9eee690d39ec8fed0c2c WatchSource:0}: Error finding container 14fb6b7ba770140322c060966d55bd272d29e3d265dc9eee690d39ec8fed0c2c: Status 404 returned error can't find the container with id 14fb6b7ba770140322c060966d55bd272d29e3d265dc9eee690d39ec8fed0c2c
Jul 02 19:13:45.223449 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:45.223370    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6360c37a-f946-46e8-a1fd-6347e3064d4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:45.225809 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:45.225772    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6360c37a-f946-46e8-a1fd-6347e3064d4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pdmlw\" (UID: \"6360c37a-f946-46e8-a1fd-6347e3064d4d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:45.250801 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:45.250772    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28" event={"ID":"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6","Type":"ContainerStarted","Data":"14fb6b7ba770140322c060966d55bd272d29e3d265dc9eee690d39ec8fed0c2c"}
Jul 02 19:13:45.498137 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:45.498066    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"
Jul 02 19:13:45.828347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:45.828318    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw"]
Jul 02 19:13:45.836079 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:13:45.836050    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6360c37a_f946_46e8_a1fd_6347e3064d4d.slice/crio-4ab700a3c80278e4ffefd084201364a1ae3a199f05e96b7e6c916840346576a6 WatchSource:0}: Error finding container 4ab700a3c80278e4ffefd084201364a1ae3a199f05e96b7e6c916840346576a6: Status 404 returned error can't find the container with id 4ab700a3c80278e4ffefd084201364a1ae3a199f05e96b7e6c916840346576a6
Jul 02 19:13:46.257799 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:46.257756    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw" event={"ID":"6360c37a-f946-46e8-a1fd-6347e3064d4d","Type":"ContainerStarted","Data":"4ab700a3c80278e4ffefd084201364a1ae3a199f05e96b7e6c916840346576a6"}
Jul 02 19:13:47.266752 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:47.266716    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28" event={"ID":"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6","Type":"ContainerStarted","Data":"8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38"}
Jul 02 19:13:47.267229 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:47.266830    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:47.297908 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:47.297789    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28" podStartSLOduration=1.5251319859999999 podStartE2EDuration="3.297768511s" totalImagesPullingTime="1.772636525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:47.295275538 +0000 UTC m=+508.445307521" watchObservedRunningTime="2026-07-02 19:13:47.297768511 +0000 UTC m=+508.447800524"
Jul 02 19:13:53.243645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:53.243609    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:54.989880 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:54.989620    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"]
Jul 02 19:13:54.990524 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:54.990118    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" containerName="manager" containerID="cri-o://4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b" gracePeriod=2
Jul 02 19:13:54.992812 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:54.992772    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"]
Jul 02 19:13:54.992812 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:54.992795    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:55.008641 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.008613    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"]
Jul 02 19:13:55.008886 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.008857    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" containerName="manager" containerID="cri-o://8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38" gracePeriod=2
Jul 02 19:13:55.012513 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.011280    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:55.012629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.012566    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:55.019224 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.017888    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"]
Jul 02 19:13:55.020394 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.020367    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"]
Jul 02 19:13:55.020865 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.020844    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" containerName="manager"
Jul 02 19:13:55.020963 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.020868    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" containerName="manager"
Jul 02 19:13:55.020963 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.020896    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" containerName="manager"
Jul 02 19:13:55.020963 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.020904    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" containerName="manager"
Jul 02 19:13:55.021125 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.020987    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" containerName="manager"
Jul 02 19:13:55.021125 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.021001    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" containerName="manager"
Jul 02 19:13:55.022073 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.022038    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:55.024457 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.024440    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.024644 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.024473    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:55.034602 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.034574    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"]
Jul 02 19:13:55.040688 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.040667    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"]
Jul 02 19:13:55.040788 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.040773    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:13:55.056102 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.056074    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"]
Jul 02 19:13:55.066918 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.066865    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:55.069305 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.069248    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:55.071629 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.071595    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:55.096552 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.096515    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:55.113996 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.113945    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtrh\" (UniqueName: \"kubernetes.io/projected/20925237-849c-4aaa-8ee4-50eef5fd1e0e-kube-api-access-tjtrh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2j57h\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.114189 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.114048    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmslb\" (UniqueName: \"kubernetes.io/projected/bc010be6-4f46-4ab8-b97c-b8b3e4734ae1-kube-api-access-rmslb\") pod \"limitador-operator-controller-manager-859d7b78bd-jq5lr\" (UID: \"bc010be6-4f46-4ab8-b97c-b8b3e4734ae1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:13:55.114189 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.114101    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20925237-849c-4aaa-8ee4-50eef5fd1e0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2j57h\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.215310 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.215265    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtrh\" (UniqueName: \"kubernetes.io/projected/20925237-849c-4aaa-8ee4-50eef5fd1e0e-kube-api-access-tjtrh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2j57h\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.215475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.215330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmslb\" (UniqueName: \"kubernetes.io/projected/bc010be6-4f46-4ab8-b97c-b8b3e4734ae1-kube-api-access-rmslb\") pod \"limitador-operator-controller-manager-859d7b78bd-jq5lr\" (UID: \"bc010be6-4f46-4ab8-b97c-b8b3e4734ae1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:13:55.215475 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.215363    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20925237-849c-4aaa-8ee4-50eef5fd1e0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2j57h\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.215857 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.215759    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20925237-849c-4aaa-8ee4-50eef5fd1e0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2j57h\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.225732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.225702    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmslb\" (UniqueName: \"kubernetes.io/projected/bc010be6-4f46-4ab8-b97c-b8b3e4734ae1-kube-api-access-rmslb\") pod \"limitador-operator-controller-manager-859d7b78bd-jq5lr\" (UID: \"bc010be6-4f46-4ab8-b97c-b8b3e4734ae1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:13:55.232588 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.232560    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtrh\" (UniqueName: \"kubernetes.io/projected/20925237-849c-4aaa-8ee4-50eef5fd1e0e-kube-api-access-tjtrh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2j57h\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.253750 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.253652    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6t5g9"
Jul 02 19:13:55.256303 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.256271    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:13:55.258414 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:13:55.258383    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:13:55.421284 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.421237    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:13:55.427400 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:13:55.427356    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:14:06.921287 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:06.921264    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:14:06.923290 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:06.923273    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:14:06.924090 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:06.924070    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:14:06.926244 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:06.926209    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:14:06.928311 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:06.928281    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:14:07.030722 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.030647    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znxkj\" (UniqueName: \"kubernetes.io/projected/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6-kube-api-access-znxkj\") pod \"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6\" (UID: \"7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6\") "
Jul 02 19:14:07.030722 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.030682    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzsk6\" (UniqueName: \"kubernetes.io/projected/7ac78a43-7a43-416e-8ec1-ebdf29592f38-kube-api-access-gzsk6\") pod \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") "
Jul 02 19:14:07.030722 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.030705    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ac78a43-7a43-416e-8ec1-ebdf29592f38-extensions-socket-volume\") pod \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\" (UID: \"7ac78a43-7a43-416e-8ec1-ebdf29592f38\") "
Jul 02 19:14:07.031289 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.031259    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ac78a43-7a43-416e-8ec1-ebdf29592f38-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7ac78a43-7a43-416e-8ec1-ebdf29592f38" (UID: "7ac78a43-7a43-416e-8ec1-ebdf29592f38"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:14:07.032712 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.032690    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6-kube-api-access-znxkj" (OuterVolumeSpecName: "kube-api-access-znxkj") pod "7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" (UID: "7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6"). InnerVolumeSpecName "kube-api-access-znxkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:14:07.032779 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.032717    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ac78a43-7a43-416e-8ec1-ebdf29592f38-kube-api-access-gzsk6" (OuterVolumeSpecName: "kube-api-access-gzsk6") pod "7ac78a43-7a43-416e-8ec1-ebdf29592f38" (UID: "7ac78a43-7a43-416e-8ec1-ebdf29592f38"). InnerVolumeSpecName "kube-api-access-gzsk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:14:07.120732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.120688    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"]
Jul 02 19:14:07.122894 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:14:07.122853    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc010be6_4f46_4ab8_b97c_b8b3e4734ae1.slice/crio-86acc6884741747214d4fd8a1f883d4de25dd9a0bfed6c14981f9282620eb660 WatchSource:0}: Error finding container 86acc6884741747214d4fd8a1f883d4de25dd9a0bfed6c14981f9282620eb660: Status 404 returned error can't find the container with id 86acc6884741747214d4fd8a1f883d4de25dd9a0bfed6c14981f9282620eb660
Jul 02 19:14:07.123299 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.123280    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"]
Jul 02 19:14:07.125196 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:14:07.125174    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20925237_849c_4aaa_8ee4_50eef5fd1e0e.slice/crio-9baba25617cf9b1f4313f5563559aec2f4e421cabbd7fbc45184033aba4995d1 WatchSource:0}: Error finding container 9baba25617cf9b1f4313f5563559aec2f4e421cabbd7fbc45184033aba4995d1: Status 404 returned error can't find the container with id 9baba25617cf9b1f4313f5563559aec2f4e421cabbd7fbc45184033aba4995d1
Jul 02 19:14:07.132112 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.132090    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znxkj\" (UniqueName: \"kubernetes.io/projected/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6-kube-api-access-znxkj\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:07.132203 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.132114    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzsk6\" (UniqueName: \"kubernetes.io/projected/7ac78a43-7a43-416e-8ec1-ebdf29592f38-kube-api-access-gzsk6\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:07.132203 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.132126    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ac78a43-7a43-416e-8ec1-ebdf29592f38-extensions-socket-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:07.314595 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.314520    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" path="/var/lib/kubelet/pods/7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6/volumes"
Jul 02 19:14:07.314870 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.314858    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" path="/var/lib/kubelet/pods/7ac78a43-7a43-416e-8ec1-ebdf29592f38/volumes"
Jul 02 19:14:07.368774 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.368741    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr" event={"ID":"bc010be6-4f46-4ab8-b97c-b8b3e4734ae1","Type":"ContainerStarted","Data":"63858dd123b9ffaf959d36123dd817cd6aef8c55cf686e559b58a25fe44b9243"}
Jul 02 19:14:07.368955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.368803    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr" event={"ID":"bc010be6-4f46-4ab8-b97c-b8b3e4734ae1","Type":"ContainerStarted","Data":"86acc6884741747214d4fd8a1f883d4de25dd9a0bfed6c14981f9282620eb660"}
Jul 02 19:14:07.368955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.368831    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:14:07.369938 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.369913    2585 generic.go:356] "Generic (PLEG): container finished" podID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" containerID="8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38" exitCode=0
Jul 02 19:14:07.370070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.369999    2585 scope.go:117] "RemoveContainer" containerID="8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38"
Jul 02 19:14:07.370070 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.370012    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:14:07.371620 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.371592    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw" event={"ID":"6360c37a-f946-46e8-a1fd-6347e3064d4d","Type":"ContainerStarted","Data":"b8ba08b8095fe74a14ff74faa6bab4d30dc3268b27b27486868f7c745faff728"}
Jul 02 19:14:07.373121 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.373103    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" containerID="4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b" exitCode=0
Jul 02 19:14:07.373245 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.373137    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:14:07.374884 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.374864    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h" event={"ID":"20925237-849c-4aaa-8ee4-50eef5fd1e0e","Type":"ContainerStarted","Data":"e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5"}
Jul 02 19:14:07.374997 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.374907    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h" event={"ID":"20925237-849c-4aaa-8ee4-50eef5fd1e0e","Type":"ContainerStarted","Data":"9baba25617cf9b1f4313f5563559aec2f4e421cabbd7fbc45184033aba4995d1"}
Jul 02 19:14:07.374997 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.374948    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:14:07.379179 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.379163    2585 scope.go:117] "RemoveContainer" containerID="8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38"
Jul 02 19:14:07.379414 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:07.379398    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38\": container with ID starting with 8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38 not found: ID does not exist" containerID="8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38"
Jul 02 19:14:07.379479 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.379422    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38"} err="failed to get container status \"8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38\": rpc error: code = NotFound desc = could not find container \"8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38\": container with ID starting with 8031d81d993293bed0c15fc82df9c3dbc32ab909750d7e96673924f191a6be38 not found: ID does not exist"
Jul 02 19:14:07.379479 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.379439    2585 scope.go:117] "RemoveContainer" containerID="4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b"
Jul 02 19:14:07.387004 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.386985    2585 scope.go:117] "RemoveContainer" containerID="4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b"
Jul 02 19:14:07.387067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.386971    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr" podStartSLOduration=12.386959076 podStartE2EDuration="12.386959076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:07.386256637 +0000 UTC m=+528.536288638" watchObservedRunningTime="2026-07-02 19:14:07.386959076 +0000 UTC m=+528.536991059"
Jul 02 19:14:07.387458 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:07.387439    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b\": container with ID starting with 4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b not found: ID does not exist" containerID="4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b"
Jul 02 19:14:07.387515 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.387466    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b"} err="failed to get container status \"4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b\": rpc error: code = NotFound desc = could not find container \"4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b\": container with ID starting with 4b272d81bc611a65e02726f275e6878b79909f7f09ec92315f1a91c83ebe4e0b not found: ID does not exist"
Jul 02 19:14:07.407061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.407020    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h" podStartSLOduration=13.407008787 podStartE2EDuration="13.407008787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:07.406393306 +0000 UTC m=+528.556425318" watchObservedRunningTime="2026-07-02 19:14:07.407008787 +0000 UTC m=+528.557040830"
Jul 02 19:14:07.409143 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:07.409115    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:14:07.411066 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:07.411048    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:14:07.429131 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:07.429093    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pdmlw" podStartSLOduration=2.421044606 podStartE2EDuration="23.429080096s" totalImagesPullingTime="21.00803549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:07.427812535 +0000 UTC m=+528.577844519" watchObservedRunningTime="2026-07-02 19:14:07.429080096 +0000 UTC m=+528.579112079"
Jul 02 19:14:09.313535 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:09.313499    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gqh28\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7a1d37bd-40fc-4c9e-8a8a-9a1a74d733f6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gqh28"
Jul 02 19:14:09.315517 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:09.315491    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ghxp2\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="7ac78a43-7a43-416e-8ec1-ebdf29592f38" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ghxp2"
Jul 02 19:14:13.144189 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.144088    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"]
Jul 02 19:14:13.144603 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.144300    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h" podUID="20925237-849c-4aaa-8ee4-50eef5fd1e0e" containerName="manager" containerID="cri-o://e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5" gracePeriod=10
Jul 02 19:14:13.146260 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.146237    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:14:13.391659 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.391638    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:14:13.405105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.405041    2585 generic.go:356] "Generic (PLEG): container finished" podID="20925237-849c-4aaa-8ee4-50eef5fd1e0e" containerID="e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5" exitCode=0
Jul 02 19:14:13.405105 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.405083    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"
Jul 02 19:14:13.405284 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.405128    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h" event={"ID":"20925237-849c-4aaa-8ee4-50eef5fd1e0e","Type":"ContainerDied","Data":"e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5"}
Jul 02 19:14:13.405284 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.405194    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h" event={"ID":"20925237-849c-4aaa-8ee4-50eef5fd1e0e","Type":"ContainerDied","Data":"9baba25617cf9b1f4313f5563559aec2f4e421cabbd7fbc45184033aba4995d1"}
Jul 02 19:14:13.405284 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.405219    2585 scope.go:117] "RemoveContainer" containerID="e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5"
Jul 02 19:14:13.413234 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.413215    2585 scope.go:117] "RemoveContainer" containerID="e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5"
Jul 02 19:14:13.413487 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:13.413463    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5\": container with ID starting with e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5 not found: ID does not exist" containerID="e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5"
Jul 02 19:14:13.413546 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.413500    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5"} err="failed to get container status \"e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5\": rpc error: code = NotFound desc = could not find container \"e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5\": container with ID starting with e723dd0a1b8a75486d21f182947bbc3ae18537af911ee640aed5af8772f8ccb5 not found: ID does not exist"
Jul 02 19:14:13.489321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.489293    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjtrh\" (UniqueName: \"kubernetes.io/projected/20925237-849c-4aaa-8ee4-50eef5fd1e0e-kube-api-access-tjtrh\") pod \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") "
Jul 02 19:14:13.489321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.489326    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20925237-849c-4aaa-8ee4-50eef5fd1e0e-extensions-socket-volume\") pod \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\" (UID: \"20925237-849c-4aaa-8ee4-50eef5fd1e0e\") "
Jul 02 19:14:13.489725 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.489697    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20925237-849c-4aaa-8ee4-50eef5fd1e0e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "20925237-849c-4aaa-8ee4-50eef5fd1e0e" (UID: "20925237-849c-4aaa-8ee4-50eef5fd1e0e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:14:13.491323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.491303    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20925237-849c-4aaa-8ee4-50eef5fd1e0e-kube-api-access-tjtrh" (OuterVolumeSpecName: "kube-api-access-tjtrh") pod "20925237-849c-4aaa-8ee4-50eef5fd1e0e" (UID: "20925237-849c-4aaa-8ee4-50eef5fd1e0e"). InnerVolumeSpecName "kube-api-access-tjtrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:14:13.590206 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.590142    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjtrh\" (UniqueName: \"kubernetes.io/projected/20925237-849c-4aaa-8ee4-50eef5fd1e0e-kube-api-access-tjtrh\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:13.590206 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.590200    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20925237-849c-4aaa-8ee4-50eef5fd1e0e-extensions-socket-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:13.729957 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.729927    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"]
Jul 02 19:14:13.735924 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:13.735899    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2j57h"]
Jul 02 19:14:15.314488 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:15.314457    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20925237-849c-4aaa-8ee4-50eef5fd1e0e" path="/var/lib/kubelet/pods/20925237-849c-4aaa-8ee4-50eef5fd1e0e/volumes"
Jul 02 19:14:18.383714 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:18.383682    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq5lr"
Jul 02 19:14:29.465099 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.465034    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"]
Jul 02 19:14:29.465493 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.465440    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20925237-849c-4aaa-8ee4-50eef5fd1e0e" containerName="manager"
Jul 02 19:14:29.465493 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.465455    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="20925237-849c-4aaa-8ee4-50eef5fd1e0e" containerName="manager"
Jul 02 19:14:29.465567 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.465526    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="20925237-849c-4aaa-8ee4-50eef5fd1e0e" containerName="manager"
Jul 02 19:14:29.580257 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.580221    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"]
Jul 02 19:14:29.580407 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.580350    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.583063 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.583043    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jvxrn\""
Jul 02 19:14:29.623942 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.623917    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.623942 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.623949    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624131 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.623992    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624131 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.624017    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624131 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.624059    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.624122    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.624187    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.624224    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77nz\" (UniqueName: \"kubernetes.io/projected/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-kube-api-access-c77nz\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.624294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.624266    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.724831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724762    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.724831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.724831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724821    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c77nz\" (UniqueName: \"kubernetes.io/projected/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-kube-api-access-c77nz\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725068 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724865    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725068 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724889    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725068 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725068 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724956    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725068 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.724998    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725068 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.725024    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725338 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.725193    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725338 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.725307    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725411 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.725394    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725529 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.725502    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.725790 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.725766    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.727308 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.727281    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.727508 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.727491    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.733004 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.732980    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.733248 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.733230    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77nz\" (UniqueName: \"kubernetes.io/projected/b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c-kube-api-access-c77nz\") pod \"maas-default-gateway-openshift-default-56654dcf7-s6kxs\" (UID: \"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:29.891675 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:29.891643    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:30.064362 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.064330    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"]
Jul 02 19:14:30.068223 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:14:30.068195    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4448b2b_d5f9_48e1_9c63_50f09ddbbd8c.slice/crio-f573396ce08ba840510293c29a509a2d3227032dde79505dbeab18c8ffca4e1f WatchSource:0}: Error finding container f573396ce08ba840510293c29a509a2d3227032dde79505dbeab18c8ffca4e1f: Status 404 returned error can't find the container with id f573396ce08ba840510293c29a509a2d3227032dde79505dbeab18c8ffca4e1f
Jul 02 19:14:30.070485 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.070448    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:14:30.070580 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.070519    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:14:30.070580 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.070547    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:14:30.487014 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.486981    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" event={"ID":"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c","Type":"ContainerStarted","Data":"d34430b0da2b5da12fb0f40524def5e7b85b613eff73fa74534d7096970ba0d5"}
Jul 02 19:14:30.487444 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.487040    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" event={"ID":"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c","Type":"ContainerStarted","Data":"f573396ce08ba840510293c29a509a2d3227032dde79505dbeab18c8ffca4e1f"}
Jul 02 19:14:30.511723 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.511679    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" podStartSLOduration=1.511662742 podStartE2EDuration="1.511662742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:30.509399102 +0000 UTC m=+551.659431085" watchObservedRunningTime="2026-07-02 19:14:30.511662742 +0000 UTC m=+551.661694724"
Jul 02 19:14:30.892395 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:30.892369    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:31.569042 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.569008    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:31.592902 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.592877    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:31.593040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.592964    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.595699 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.595676    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 02 19:14:31.637984 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.637953    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxddw\" (UniqueName: \"kubernetes.io/projected/4d80cb07-353a-45d5-9357-f59a53bae284-kube-api-access-cxddw\") pod \"limitador-limitador-64c94fd64f-44gwz\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.638126 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.638010    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d80cb07-353a-45d5-9357-f59a53bae284-config-file\") pod \"limitador-limitador-64c94fd64f-44gwz\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.668013 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.667971    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:31.738765 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.738732    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxddw\" (UniqueName: \"kubernetes.io/projected/4d80cb07-353a-45d5-9357-f59a53bae284-kube-api-access-cxddw\") pod \"limitador-limitador-64c94fd64f-44gwz\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.738917 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.738782    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d80cb07-353a-45d5-9357-f59a53bae284-config-file\") pod \"limitador-limitador-64c94fd64f-44gwz\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.739439 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.739416    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d80cb07-353a-45d5-9357-f59a53bae284-config-file\") pod \"limitador-limitador-64c94fd64f-44gwz\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.747435 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.747414    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxddw\" (UniqueName: \"kubernetes.io/projected/4d80cb07-353a-45d5-9357-f59a53bae284-kube-api-access-cxddw\") pod \"limitador-limitador-64c94fd64f-44gwz\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:31.897513 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.897484    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:31.903382 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:31.903362    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:32.048112 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:32.048091    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:32.050424 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:14:32.050394    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d80cb07_353a_45d5_9357_f59a53bae284.slice/crio-0769d9863c4766b11e7097bbb0f00f52c71e2fc954b231e4dd1bb101730bfabf WatchSource:0}: Error finding container 0769d9863c4766b11e7097bbb0f00f52c71e2fc954b231e4dd1bb101730bfabf: Status 404 returned error can't find the container with id 0769d9863c4766b11e7097bbb0f00f52c71e2fc954b231e4dd1bb101730bfabf
Jul 02 19:14:32.496314 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:32.496279    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz" event={"ID":"4d80cb07-353a-45d5-9357-f59a53bae284","Type":"ContainerStarted","Data":"0769d9863c4766b11e7097bbb0f00f52c71e2fc954b231e4dd1bb101730bfabf"}
Jul 02 19:14:32.496528 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:32.496510    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:32.497493 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:32.497473    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:14:35.515144 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:35.515111    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz" event={"ID":"4d80cb07-353a-45d5-9357-f59a53bae284","Type":"ContainerStarted","Data":"b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8"}
Jul 02 19:14:35.515550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:35.515211    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:35.531491 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:35.531449    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz" podStartSLOduration=1.187135226 podStartE2EDuration="4.531436634s" totalImagesPullingTime="3.344301408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:35.530111002 +0000 UTC m=+556.680142985" watchObservedRunningTime="2026-07-02 19:14:35.531436634 +0000 UTC m=+556.681468617"
Jul 02 19:14:46.519899 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:46.519866    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:47.255883 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.255847    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:47.256098 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.256073    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz" podUID="4d80cb07-353a-45d5-9357-f59a53bae284" containerName="limitador" containerID="cri-o://b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8" gracePeriod=30
Jul 02 19:14:47.806624 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.806599    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:47.877176 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.877087    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxddw\" (UniqueName: \"kubernetes.io/projected/4d80cb07-353a-45d5-9357-f59a53bae284-kube-api-access-cxddw\") pod \"4d80cb07-353a-45d5-9357-f59a53bae284\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") "
Jul 02 19:14:47.877311 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.877240    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d80cb07-353a-45d5-9357-f59a53bae284-config-file\") pod \"4d80cb07-353a-45d5-9357-f59a53bae284\" (UID: \"4d80cb07-353a-45d5-9357-f59a53bae284\") "
Jul 02 19:14:47.877585 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.877558    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d80cb07-353a-45d5-9357-f59a53bae284-config-file" (OuterVolumeSpecName: "config-file") pod "4d80cb07-353a-45d5-9357-f59a53bae284" (UID: "4d80cb07-353a-45d5-9357-f59a53bae284"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:14:47.879219 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.879200    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d80cb07-353a-45d5-9357-f59a53bae284-kube-api-access-cxddw" (OuterVolumeSpecName: "kube-api-access-cxddw") pod "4d80cb07-353a-45d5-9357-f59a53bae284" (UID: "4d80cb07-353a-45d5-9357-f59a53bae284"). InnerVolumeSpecName "kube-api-access-cxddw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:14:47.978614 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.978586    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxddw\" (UniqueName: \"kubernetes.io/projected/4d80cb07-353a-45d5-9357-f59a53bae284-kube-api-access-cxddw\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:47.978614 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:47.978611    2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d80cb07-353a-45d5-9357-f59a53bae284-config-file\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:14:48.572297 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.572260    2585 generic.go:356] "Generic (PLEG): container finished" podID="4d80cb07-353a-45d5-9357-f59a53bae284" containerID="b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8" exitCode=0
Jul 02 19:14:48.572481 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.572304    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz"
Jul 02 19:14:48.572481 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.572349    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz" event={"ID":"4d80cb07-353a-45d5-9357-f59a53bae284","Type":"ContainerDied","Data":"b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8"}
Jul 02 19:14:48.572481 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.572395    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-44gwz" event={"ID":"4d80cb07-353a-45d5-9357-f59a53bae284","Type":"ContainerDied","Data":"0769d9863c4766b11e7097bbb0f00f52c71e2fc954b231e4dd1bb101730bfabf"}
Jul 02 19:14:48.572481 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.572420    2585 scope.go:117] "RemoveContainer" containerID="b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8"
Jul 02 19:14:48.580950 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.580929    2585 scope.go:117] "RemoveContainer" containerID="b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8"
Jul 02 19:14:48.581203 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:14:48.581183    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8\": container with ID starting with b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8 not found: ID does not exist" containerID="b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8"
Jul 02 19:14:48.581276 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.581211    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8"} err="failed to get container status \"b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8\": rpc error: code = NotFound desc = could not find container \"b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8\": container with ID starting with b7b5f1fe108f9f6adf6dc20606a6a499e9ddd3969cc66e391eecae6a78d307b8 not found: ID does not exist"
Jul 02 19:14:48.593772 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.593750    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:48.600831 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:48.600811    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-44gwz"]
Jul 02 19:14:49.314927 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:49.314893    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d80cb07-353a-45d5-9357-f59a53bae284" path="/var/lib/kubelet/pods/4d80cb07-353a-45d5-9357-f59a53bae284/volumes"
Jul 02 19:14:53.074411 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.074381    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-wwczn"]
Jul 02 19:14:53.074885 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.074867    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d80cb07-353a-45d5-9357-f59a53bae284" containerName="limitador"
Jul 02 19:14:53.074960 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.074888    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d80cb07-353a-45d5-9357-f59a53bae284" containerName="limitador"
Jul 02 19:14:53.075011 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.074981    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d80cb07-353a-45d5-9357-f59a53bae284" containerName="limitador"
Jul 02 19:14:53.079389 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.079368    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.081986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.081966    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 02 19:14:53.081986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.081981    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c9txk\""
Jul 02 19:14:53.085910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.085888    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-wwczn"]
Jul 02 19:14:53.124210 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.124187    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/45fdd633-bf3a-4902-b929-0ed89c4c290c-data\") pod \"postgres-69fb4594cf-wwczn\" (UID: \"45fdd633-bf3a-4902-b929-0ed89c4c290c\") " pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.124342 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.124231    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlkdv\" (UniqueName: \"kubernetes.io/projected/45fdd633-bf3a-4902-b929-0ed89c4c290c-kube-api-access-dlkdv\") pod \"postgres-69fb4594cf-wwczn\" (UID: \"45fdd633-bf3a-4902-b929-0ed89c4c290c\") " pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.225347 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.225313    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/45fdd633-bf3a-4902-b929-0ed89c4c290c-data\") pod \"postgres-69fb4594cf-wwczn\" (UID: \"45fdd633-bf3a-4902-b929-0ed89c4c290c\") " pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.225508 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.225358    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlkdv\" (UniqueName: \"kubernetes.io/projected/45fdd633-bf3a-4902-b929-0ed89c4c290c-kube-api-access-dlkdv\") pod \"postgres-69fb4594cf-wwczn\" (UID: \"45fdd633-bf3a-4902-b929-0ed89c4c290c\") " pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.225669 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.225651    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/45fdd633-bf3a-4902-b929-0ed89c4c290c-data\") pod \"postgres-69fb4594cf-wwczn\" (UID: \"45fdd633-bf3a-4902-b929-0ed89c4c290c\") " pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.233580 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.233559    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlkdv\" (UniqueName: \"kubernetes.io/projected/45fdd633-bf3a-4902-b929-0ed89c4c290c-kube-api-access-dlkdv\") pod \"postgres-69fb4594cf-wwczn\" (UID: \"45fdd633-bf3a-4902-b929-0ed89c4c290c\") " pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.393022 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.393002    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:53.541310 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.541287    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-wwczn"]
Jul 02 19:14:53.544083 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:14:53.544056    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fdd633_bf3a_4902_b929_0ed89c4c290c.slice/crio-70d81ecbddf321b7d40c00ad34396be756b3c944a963e67ab6fcee5ed3c7ec37 WatchSource:0}: Error finding container 70d81ecbddf321b7d40c00ad34396be756b3c944a963e67ab6fcee5ed3c7ec37: Status 404 returned error can't find the container with id 70d81ecbddf321b7d40c00ad34396be756b3c944a963e67ab6fcee5ed3c7ec37
Jul 02 19:14:53.594902 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:53.594876    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-wwczn" event={"ID":"45fdd633-bf3a-4902-b929-0ed89c4c290c","Type":"ContainerStarted","Data":"70d81ecbddf321b7d40c00ad34396be756b3c944a963e67ab6fcee5ed3c7ec37"}
Jul 02 19:14:59.627043 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:59.626959    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-wwczn" event={"ID":"45fdd633-bf3a-4902-b929-0ed89c4c290c","Type":"ContainerStarted","Data":"2363d440c5adb047e5cef8e019a915125f83c670a35919855f0bd73d22da444d"}
Jul 02 19:14:59.627549 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:59.627098    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:14:59.643698 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:14:59.643640    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-wwczn" podStartSLOduration=0.958300461 podStartE2EDuration="6.643619226s" totalImagesPullingTime="5.685318765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:59.641520095 +0000 UTC m=+580.791552079" watchObservedRunningTime="2026-07-02 19:14:59.643619226 +0000 UTC m=+580.793651211"
Jul 02 19:15:05.661295 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:05.661268    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-wwczn"
Jul 02 19:15:12.808167 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.808113    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8db8849b9-q5sxc"]
Jul 02 19:15:12.814928 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.814907    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:12.817671 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.817647    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 02 19:15:12.817772 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.817672    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5jqvs\""
Jul 02 19:15:12.818700 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.818686    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 02 19:15:12.824524 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.824498    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8db8849b9-q5sxc"]
Jul 02 19:15:12.900618 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.900589    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:12.900778 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.900644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzb68\" (UniqueName: \"kubernetes.io/projected/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-kube-api-access-jzb68\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:12.983769 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.983735    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8fd569cc8-x8lgr"]
Jul 02 19:15:12.987354 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.987333    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:12.998180 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:12.998140    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-x8lgr"]
Jul 02 19:15:13.001415 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.001392    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzb68\" (UniqueName: \"kubernetes.io/projected/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-kube-api-access-jzb68\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:13.001669 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.001467    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:13.001669 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:15:13.001578    2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 02 19:15:13.001669 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:15:13.001650    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert podName:dfb97eb2-a51a-4226-a28b-16dc8d7b91fe nodeName:}" failed. No retries permitted until 2026-07-02 19:15:13.501628468 +0000 UTC m=+594.651660432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert") pod "maas-controller-8db8849b9-q5sxc" (UID: "dfb97eb2-a51a-4226-a28b-16dc8d7b91fe") : secret "maas-controller-webhook-cert" not found
Jul 02 19:15:13.010062 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.010037    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzb68\" (UniqueName: \"kubernetes.io/projected/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-kube-api-access-jzb68\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:13.102415 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.102340    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5cmw\" (UniqueName: \"kubernetes.io/projected/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-kube-api-access-s5cmw\") pod \"maas-controller-8fd569cc8-x8lgr\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") " pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.102553 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.102422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-webhook-cert\") pod \"maas-controller-8fd569cc8-x8lgr\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") " pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.203877 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.203845    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cmw\" (UniqueName: \"kubernetes.io/projected/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-kube-api-access-s5cmw\") pod \"maas-controller-8fd569cc8-x8lgr\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") " pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.204013 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.203906    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-webhook-cert\") pod \"maas-controller-8fd569cc8-x8lgr\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") " pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.206420 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.206400    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-webhook-cert\") pod \"maas-controller-8fd569cc8-x8lgr\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") " pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.212108 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.212087    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5cmw\" (UniqueName: \"kubernetes.io/projected/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-kube-api-access-s5cmw\") pod \"maas-controller-8fd569cc8-x8lgr\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") " pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.298290 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.298257    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:13.434372 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.434347    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-x8lgr"]
Jul 02 19:15:13.436175 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:15:13.436136    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d91bfaa_f50b_4b2c_b824_53c33a4c3618.slice/crio-d16cf28c0dcf5a50a4642a9cdeddf73be96515d5676768acbcffe87960e338ae WatchSource:0}: Error finding container d16cf28c0dcf5a50a4642a9cdeddf73be96515d5676768acbcffe87960e338ae: Status 404 returned error can't find the container with id d16cf28c0dcf5a50a4642a9cdeddf73be96515d5676768acbcffe87960e338ae
Jul 02 19:15:13.506164 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.506124    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:13.508450 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.508429    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert\") pod \"maas-controller-8db8849b9-q5sxc\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") " pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:13.689360 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.689325    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8fd569cc8-x8lgr" event={"ID":"6d91bfaa-f50b-4b2c-b824-53c33a4c3618","Type":"ContainerStarted","Data":"d16cf28c0dcf5a50a4642a9cdeddf73be96515d5676768acbcffe87960e338ae"}
Jul 02 19:15:13.726637 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.726594    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:13.857483 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:13.857461    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8db8849b9-q5sxc"]
Jul 02 19:15:13.859469 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:15:13.859438    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb97eb2_a51a_4226_a28b_16dc8d7b91fe.slice/crio-5af8ba4f60e70f1d4363ccbb6ef70b886d59644404be040bfee0150623267248 WatchSource:0}: Error finding container 5af8ba4f60e70f1d4363ccbb6ef70b886d59644404be040bfee0150623267248: Status 404 returned error can't find the container with id 5af8ba4f60e70f1d4363ccbb6ef70b886d59644404be040bfee0150623267248
Jul 02 19:15:14.698218 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:14.698173    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8db8849b9-q5sxc" event={"ID":"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe","Type":"ContainerStarted","Data":"5af8ba4f60e70f1d4363ccbb6ef70b886d59644404be040bfee0150623267248"}
Jul 02 19:15:16.709371 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:16.709336    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8fd569cc8-x8lgr" event={"ID":"6d91bfaa-f50b-4b2c-b824-53c33a4c3618","Type":"ContainerStarted","Data":"c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9"}
Jul 02 19:15:16.709874 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:16.709523    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:16.710805 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:16.710782    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8db8849b9-q5sxc" event={"ID":"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe","Type":"ContainerStarted","Data":"2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2"}
Jul 02 19:15:16.710901 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:16.710889    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:16.754549 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:16.754504    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8fd569cc8-x8lgr" podStartSLOduration=2.118277568 podStartE2EDuration="4.754490674s" totalImagesPullingTime="2.636213106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:16.753217314 +0000 UTC m=+597.903249298" watchObservedRunningTime="2026-07-02 19:15:16.754490674 +0000 UTC m=+597.904522656"
Jul 02 19:15:16.781870 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:16.781820    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8db8849b9-q5sxc" podStartSLOduration=2.558963633 podStartE2EDuration="4.781804848s" totalImagesPullingTime="2.222841215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:16.779991711 +0000 UTC m=+597.930023686" watchObservedRunningTime="2026-07-02 19:15:16.781804848 +0000 UTC m=+597.931836832"
Jul 02 19:15:27.719719 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:27.719688    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:15:27.720210 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:27.719771    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:27.766406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:27.766374    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8db8849b9-q5sxc"]
Jul 02 19:15:27.766669 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:27.766591    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-8db8849b9-q5sxc" podUID="dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" containerName="manager" containerID="cri-o://2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2" gracePeriod=30
Jul 02 19:15:28.024532 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.024507    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:28.037949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.037930    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert\") pod \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") "
Jul 02 19:15:28.038056 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.037975    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzb68\" (UniqueName: \"kubernetes.io/projected/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-kube-api-access-jzb68\") pod \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\" (UID: \"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe\") "
Jul 02 19:15:28.039914 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.039890    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" (UID: "dfb97eb2-a51a-4226-a28b-16dc8d7b91fe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:15:28.040013 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.039988    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-kube-api-access-jzb68" (OuterVolumeSpecName: "kube-api-access-jzb68") pod "dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" (UID: "dfb97eb2-a51a-4226-a28b-16dc8d7b91fe"). InnerVolumeSpecName "kube-api-access-jzb68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:15:28.139517 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.139482    2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-webhook-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:15:28.139517 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.139519    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzb68\" (UniqueName: \"kubernetes.io/projected/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe-kube-api-access-jzb68\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:15:28.765888 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.765855    2585 generic.go:356] "Generic (PLEG): container finished" podID="dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" containerID="2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2" exitCode=0
Jul 02 19:15:28.766296 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.765896    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8db8849b9-q5sxc"
Jul 02 19:15:28.766296 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.765926    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8db8849b9-q5sxc" event={"ID":"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe","Type":"ContainerDied","Data":"2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2"}
Jul 02 19:15:28.766296 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.765970    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8db8849b9-q5sxc" event={"ID":"dfb97eb2-a51a-4226-a28b-16dc8d7b91fe","Type":"ContainerDied","Data":"5af8ba4f60e70f1d4363ccbb6ef70b886d59644404be040bfee0150623267248"}
Jul 02 19:15:28.766296 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.765993    2585 scope.go:117] "RemoveContainer" containerID="2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2"
Jul 02 19:15:28.774605 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.774589    2585 scope.go:117] "RemoveContainer" containerID="2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2"
Jul 02 19:15:28.774829 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:15:28.774812    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2\": container with ID starting with 2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2 not found: ID does not exist" containerID="2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2"
Jul 02 19:15:28.774878 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.774839    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2"} err="failed to get container status \"2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2\": rpc error: code = NotFound desc = could not find container \"2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2\": container with ID starting with 2cb943c8fa8ed82091c9f2d82eada8ed243d56df424cbbd2d5ec4ae6fca94dd2 not found: ID does not exist"
Jul 02 19:15:28.787339 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.787314    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8db8849b9-q5sxc"]
Jul 02 19:15:28.793359 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:28.793339    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8db8849b9-q5sxc"]
Jul 02 19:15:29.314959 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:29.314930    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" path="/var/lib/kubelet/pods/dfb97eb2-a51a-4226-a28b-16dc8d7b91fe/volumes"
Jul 02 19:15:46.265038 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.264962    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-57b5b68ff8-shtlb"]
Jul 02 19:15:46.265497 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.265341    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" containerName="manager"
Jul 02 19:15:46.265497 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.265359    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" containerName="manager"
Jul 02 19:15:46.265497 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.265430    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfb97eb2-a51a-4226-a28b-16dc8d7b91fe" containerName="manager"
Jul 02 19:15:46.268461 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.268447    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.271128 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.271110    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-ftzfx\""
Jul 02 19:15:46.272017 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.272002    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 02 19:15:46.282431 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.282403    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-57b5b68ff8-shtlb"]
Jul 02 19:15:46.404539 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.404502    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93804015-97b3-4685-9223-97788020b3d1-openshift-service-ca\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.404717 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.404564    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93804015-97b3-4685-9223-97788020b3d1-maas-api-tls\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.404717 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.404669    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc96v\" (UniqueName: \"kubernetes.io/projected/93804015-97b3-4685-9223-97788020b3d1-kube-api-access-gc96v\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.505393 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.505354    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc96v\" (UniqueName: \"kubernetes.io/projected/93804015-97b3-4685-9223-97788020b3d1-kube-api-access-gc96v\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.505582 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.505425    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93804015-97b3-4685-9223-97788020b3d1-openshift-service-ca\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.505582 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.505473    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93804015-97b3-4685-9223-97788020b3d1-maas-api-tls\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.506131 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.506096    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93804015-97b3-4685-9223-97788020b3d1-openshift-service-ca\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.507901 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.507878    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93804015-97b3-4685-9223-97788020b3d1-maas-api-tls\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.514549 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.514522    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc96v\" (UniqueName: \"kubernetes.io/projected/93804015-97b3-4685-9223-97788020b3d1-kube-api-access-gc96v\") pod \"maas-api-57b5b68ff8-shtlb\" (UID: \"93804015-97b3-4685-9223-97788020b3d1\") " pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.579189 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.579084    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:46.723294 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.723271    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-57b5b68ff8-shtlb"]
Jul 02 19:15:46.724886 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:15:46.724858    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93804015_97b3_4685_9223_97788020b3d1.slice/crio-c5a79f38ab47c511c12dc85da9c98310c2ff652e7995a4115ac6dba996acbc40 WatchSource:0}: Error finding container c5a79f38ab47c511c12dc85da9c98310c2ff652e7995a4115ac6dba996acbc40: Status 404 returned error can't find the container with id c5a79f38ab47c511c12dc85da9c98310c2ff652e7995a4115ac6dba996acbc40
Jul 02 19:15:46.829283 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.829217    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qmgtf"]
Jul 02 19:15:46.833694 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.833675    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:46.836284 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.836266    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 02 19:15:46.844317 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.844292    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qmgtf"]
Jul 02 19:15:46.847645 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.847622    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-57b5b68ff8-shtlb" event={"ID":"93804015-97b3-4685-9223-97788020b3d1","Type":"ContainerStarted","Data":"c5a79f38ab47c511c12dc85da9c98310c2ff652e7995a4115ac6dba996acbc40"}
Jul 02 19:15:46.908416 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.908379    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8682z\" (UniqueName: \"kubernetes.io/projected/c283369d-18b5-4752-9430-c50a4de4807f-kube-api-access-8682z\") pod \"payload-processing-58f86c7798-qmgtf\" (UID: \"c283369d-18b5-4752-9430-c50a4de4807f\") " pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:46.908565 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:46.908431    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c283369d-18b5-4752-9430-c50a4de4807f-plugins-config-volume\") pod \"payload-processing-58f86c7798-qmgtf\" (UID: \"c283369d-18b5-4752-9430-c50a4de4807f\") " pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:47.009718 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.009689    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8682z\" (UniqueName: \"kubernetes.io/projected/c283369d-18b5-4752-9430-c50a4de4807f-kube-api-access-8682z\") pod \"payload-processing-58f86c7798-qmgtf\" (UID: \"c283369d-18b5-4752-9430-c50a4de4807f\") " pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:47.009881 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.009730    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c283369d-18b5-4752-9430-c50a4de4807f-plugins-config-volume\") pod \"payload-processing-58f86c7798-qmgtf\" (UID: \"c283369d-18b5-4752-9430-c50a4de4807f\") " pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:47.010442 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.010422    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c283369d-18b5-4752-9430-c50a4de4807f-plugins-config-volume\") pod \"payload-processing-58f86c7798-qmgtf\" (UID: \"c283369d-18b5-4752-9430-c50a4de4807f\") " pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:47.021443 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.021416    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8682z\" (UniqueName: \"kubernetes.io/projected/c283369d-18b5-4752-9430-c50a4de4807f-kube-api-access-8682z\") pod \"payload-processing-58f86c7798-qmgtf\" (UID: \"c283369d-18b5-4752-9430-c50a4de4807f\") " pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:47.145938 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.145904    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:47.280107 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.279972    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qmgtf"]
Jul 02 19:15:47.282195 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:15:47.282168    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc283369d_18b5_4752_9430_c50a4de4807f.slice/crio-d7dedb22b21cc6f9a4010b1c5506971eea2686bd628c419d39ce9e477acefaa8 WatchSource:0}: Error finding container d7dedb22b21cc6f9a4010b1c5506971eea2686bd628c419d39ce9e477acefaa8: Status 404 returned error can't find the container with id d7dedb22b21cc6f9a4010b1c5506971eea2686bd628c419d39ce9e477acefaa8
Jul 02 19:15:47.854069 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:47.854038    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf" event={"ID":"c283369d-18b5-4752-9430-c50a4de4807f","Type":"ContainerStarted","Data":"d7dedb22b21cc6f9a4010b1c5506971eea2686bd628c419d39ce9e477acefaa8"}
Jul 02 19:15:49.866965 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:49.866864    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-57b5b68ff8-shtlb" event={"ID":"93804015-97b3-4685-9223-97788020b3d1","Type":"ContainerStarted","Data":"67961c3dd51d8fdf642746c753a0d17b20f2ccd3f4e1c5c284c317c233ae0e78"}
Jul 02 19:15:49.866965 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:49.866950    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:49.868555 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:49.868530    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf" event={"ID":"c283369d-18b5-4752-9430-c50a4de4807f","Type":"ContainerStarted","Data":"96653b2b3e63a0e5b536455ca82f6f3d1bf229f01f9a991e0b9d36500ccd42b2"}
Jul 02 19:15:49.868704 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:49.868590    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:15:49.884076 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:49.884025    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-57b5b68ff8-shtlb" podStartSLOduration=1.21516352 podStartE2EDuration="3.884008003s" totalImagesPullingTime="2.668844483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:49.88369225 +0000 UTC m=+631.033724234" watchObservedRunningTime="2026-07-02 19:15:49.884008003 +0000 UTC m=+631.034039988"
Jul 02 19:15:49.899139 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:49.899089    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf" podStartSLOduration=1.7861882009999999 podStartE2EDuration="3.899056834s" totalImagesPullingTime="2.112868633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:49.898631446 +0000 UTC m=+631.048663430" watchObservedRunningTime="2026-07-02 19:15:49.899056834 +0000 UTC m=+631.049088818"
Jul 02 19:15:54.893640 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:54.893607    2585 generic.go:356] "Generic (PLEG): container finished" podID="b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c" containerID="d34430b0da2b5da12fb0f40524def5e7b85b613eff73fa74534d7096970ba0d5" exitCode=0
Jul 02 19:15:54.894186 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:54.893688    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" event={"ID":"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c","Type":"ContainerDied","Data":"d34430b0da2b5da12fb0f40524def5e7b85b613eff73fa74534d7096970ba0d5"}
Jul 02 19:15:54.894265 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:54.894236    2585 scope.go:117] "RemoveContainer" containerID="d34430b0da2b5da12fb0f40524def5e7b85b613eff73fa74534d7096970ba0d5"
Jul 02 19:15:54.895283 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:54.895253    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:15:54.895388 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:54.895340    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:15:54.895452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:54.895400    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:15:55.878279 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:55.878240    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-57b5b68ff8-shtlb"
Jul 02 19:15:55.901506 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:55.901469    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" event={"ID":"b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c","Type":"ContainerStarted","Data":"b2cf357feb38adf9878d3e3c6228f6e4a066f59a6099a086b2cbfc33eb813651"}
Jul 02 19:15:56.892501 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:56.892468    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:15:56.892677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:56.892659    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-s6kxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 02 19:15:56.892731 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:56.892712    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" podUID="b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 02 19:15:57.892663 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:57.892630    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-s6kxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 02 19:15:57.893126 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:57.892699    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" podUID="b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 02 19:15:58.892902 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:58.892863    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-s6kxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 02 19:15:58.893393 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:58.892927    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs" podUID="b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 02 19:15:59.896461 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:59.896433    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:15:59.896828 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:59.896693    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:15:59.897715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:15:59.897690    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-s6kxs"
Jul 02 19:16:00.875317 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:00.875284    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-qmgtf"
Jul 02 19:16:07.483686 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.483647    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"]
Jul 02 19:16:07.487784 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.487763    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.491985 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.491960    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kkw25\""
Jul 02 19:16:07.492108 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.492003    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 02 19:16:07.492108 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.492017    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 02 19:16:07.492503 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.492484    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 02 19:16:07.497720 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.497699    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"]
Jul 02 19:16:07.515285 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.515251    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.515406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.515290    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.515406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.515346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.515406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.515372    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbv68\" (UniqueName: \"kubernetes.io/projected/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-kube-api-access-pbv68\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.515406 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.515404    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.515626 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.515420    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.616668 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.616823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616678    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.616823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616808    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.616902 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616864    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbv68\" (UniqueName: \"kubernetes.io/projected/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-kube-api-access-pbv68\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.616943 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.616985 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.617040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.616979    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.617327 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.617304    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.617433 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.617312    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.618924 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.618899    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.619109 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.619092    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.626730 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.626705    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbv68\" (UniqueName: \"kubernetes.io/projected/0f6e2d12-0136-42fa-b6f0-6e2abd39771b-kube-api-access-pbv68\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp\" (UID: \"0f6e2d12-0136-42fa-b6f0-6e2abd39771b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.799443 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.799362    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:07.937851 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.937822    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"]
Jul 02 19:16:07.939246 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:16:07.939218    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6e2d12_0136_42fa_b6f0_6e2abd39771b.slice/crio-fc73e3511bbd229165b77d9cf2ca3dced8f1763c9ceb8d08af7c358f6e91a452 WatchSource:0}: Error finding container fc73e3511bbd229165b77d9cf2ca3dced8f1763c9ceb8d08af7c358f6e91a452: Status 404 returned error can't find the container with id fc73e3511bbd229165b77d9cf2ca3dced8f1763c9ceb8d08af7c358f6e91a452
Jul 02 19:16:07.959142 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:07.959110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp" event={"ID":"0f6e2d12-0136-42fa-b6f0-6e2abd39771b","Type":"ContainerStarted","Data":"fc73e3511bbd229165b77d9cf2ca3dced8f1763c9ceb8d08af7c358f6e91a452"}
Jul 02 19:16:09.971558 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:09.971522    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp" event={"ID":"0f6e2d12-0136-42fa-b6f0-6e2abd39771b","Type":"ContainerStarted","Data":"02eabe670dc95757a5ca5be49f8fc2e7d6836552aa8e2b8cc4060518301a99da"}
Jul 02 19:16:09.991821 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:09.991774    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp" podStartSLOduration=1.8024609329999999 podStartE2EDuration="2.991758449s" totalImagesPullingTime="1.189297516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:09.989605327 +0000 UTC m=+651.139637332" watchObservedRunningTime="2026-07-02 19:16:09.991758449 +0000 UTC m=+651.141790433"
Jul 02 19:16:17.800003 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.799967    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:17.800516 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.800020    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:17.812766 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.812734    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:17.877089 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.877057    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"]
Jul 02 19:16:17.880511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.880493    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:17.883247 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.883225    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 02 19:16:17.892170 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.892133    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"]
Jul 02 19:16:17.913305 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.913279    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:17.913430 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.913336    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a7667e5-7688-4e19-b5d7-19d74e785108-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:17.913549 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.913530    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:17.913619 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.913563    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:17.913619 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.913586    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:17.913619 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:17.913609    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfp7k\" (UniqueName: \"kubernetes.io/projected/3a7667e5-7688-4e19-b5d7-19d74e785108-kube-api-access-sfp7k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014427 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014398    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014436    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a7667e5-7688-4e19-b5d7-19d74e785108-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014508    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014779 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014779 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014688    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014779 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014720    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfp7k\" (UniqueName: \"kubernetes.io/projected/3a7667e5-7688-4e19-b5d7-19d74e785108-kube-api-access-sfp7k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014936 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014878    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.014936 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.014919    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.015202 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.015182    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.016702 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.016682    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a7667e5-7688-4e19-b5d7-19d74e785108-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.016949 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.016930    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a7667e5-7688-4e19-b5d7-19d74e785108-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.021209 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.021190    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp"
Jul 02 19:16:18.023877 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.023848    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfp7k\" (UniqueName: \"kubernetes.io/projected/3a7667e5-7688-4e19-b5d7-19d74e785108-kube-api-access-sfp7k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zfn5z\" (UID: \"3a7667e5-7688-4e19-b5d7-19d74e785108\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.193556 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.193520    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:18.334971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:18.334939    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"]
Jul 02 19:16:18.336253 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:16:18.336231    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7667e5_7688_4e19_b5d7_19d74e785108.slice/crio-b2965eddf329e825e02065ad4b35414a85e2fe90b958ad095dd48411146910b6 WatchSource:0}: Error finding container b2965eddf329e825e02065ad4b35414a85e2fe90b958ad095dd48411146910b6: Status 404 returned error can't find the container with id b2965eddf329e825e02065ad4b35414a85e2fe90b958ad095dd48411146910b6
Jul 02 19:16:19.015445 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:19.015406    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z" event={"ID":"3a7667e5-7688-4e19-b5d7-19d74e785108","Type":"ContainerStarted","Data":"c598d4b31279914af35f0ca9e2f7e791eaac644ab0460566c31aff1fa5bd12b4"}
Jul 02 19:16:19.015932 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:19.015461    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z" event={"ID":"3a7667e5-7688-4e19-b5d7-19d74e785108","Type":"ContainerStarted","Data":"b2965eddf329e825e02065ad4b35414a85e2fe90b958ad095dd48411146910b6"}
Jul 02 19:16:19.041326 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:19.041277    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z" podStartSLOduration=1.784270748 podStartE2EDuration="2.041261233s" totalImagesPullingTime="256.990485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:19.03939515 +0000 UTC m=+660.189427134" watchObservedRunningTime="2026-07-02 19:16:19.041261233 +0000 UTC m=+660.191293216"
Jul 02 19:16:28.193664 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:28.193625    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:28.194140 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:28.193690    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:28.206041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:28.206021    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:29.087054 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:29.087013    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zfn5z"
Jul 02 19:16:52.890526 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.890486    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"]
Jul 02 19:16:52.894530 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.894506    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:52.897326 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.897305    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 02 19:16:52.905083 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.905062    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"]
Jul 02 19:16:52.935337 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.935305    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:52.935492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.935344    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:52.935492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.935397    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:52.935492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.935484    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:52.935649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.935528    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42c9e949-c18e-449d-80cc-6a467915ab79-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:52.935649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:52.935590    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-762lc\" (UniqueName: \"kubernetes.io/projected/42c9e949-c18e-449d-80cc-6a467915ab79-kube-api-access-762lc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.036653 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.036615    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.036653 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.036663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.036899 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.036707    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.036899 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.036740    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.036899 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.036776    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42c9e949-c18e-449d-80cc-6a467915ab79-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.036899 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.036852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-762lc\" (UniqueName: \"kubernetes.io/projected/42c9e949-c18e-449d-80cc-6a467915ab79-kube-api-access-762lc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.037168 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.037126    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.037263 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.037222    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.037582 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.037560    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.039485 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.039455    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42c9e949-c18e-449d-80cc-6a467915ab79-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.039707 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.039687    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42c9e949-c18e-449d-80cc-6a467915ab79-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.045038 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.045015    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-762lc\" (UniqueName: \"kubernetes.io/projected/42c9e949-c18e-449d-80cc-6a467915ab79-kube-api-access-762lc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s8b26\" (UID: \"42c9e949-c18e-449d-80cc-6a467915ab79\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.206275 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.206184    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:16:53.343431 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.343408    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"]
Jul 02 19:16:53.345521 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:16:53.345486    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c9e949_c18e_449d_80cc_6a467915ab79.slice/crio-bb2a408eb2108cc73f0e94801bd62bcf14d612d56bd8c6aaa431e50b9971e751 WatchSource:0}: Error finding container bb2a408eb2108cc73f0e94801bd62bcf14d612d56bd8c6aaa431e50b9971e751: Status 404 returned error can't find the container with id bb2a408eb2108cc73f0e94801bd62bcf14d612d56bd8c6aaa431e50b9971e751
Jul 02 19:16:53.346849 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:53.346825    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:16:54.196562 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:54.196524    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26" event={"ID":"42c9e949-c18e-449d-80cc-6a467915ab79","Type":"ContainerStarted","Data":"8a7f9808429194bd015c26bf7f460a62f9585bda8194294af64e630671a5edf5"}
Jul 02 19:16:54.196973 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:54.196586    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26" event={"ID":"42c9e949-c18e-449d-80cc-6a467915ab79","Type":"ContainerStarted","Data":"bb2a408eb2108cc73f0e94801bd62bcf14d612d56bd8c6aaa431e50b9971e751"}
Jul 02 19:16:54.217476 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:16:54.217419    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26" podStartSLOduration=1.958584163 podStartE2EDuration="2.217402838s" totalImagesPullingTime="258.818675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:54.215037237 +0000 UTC m=+695.365069223" watchObservedRunningTime="2026-07-02 19:16:54.217402838 +0000 UTC m=+695.367434821"
Jul 02 19:17:03.207116 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:17:03.207077    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:17:03.207590 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:17:03.207135    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:17:03.219554 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:17:03.219534    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:17:03.249653 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:17:03.249623    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s8b26"
Jul 02 19:19:04.710817 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.710733    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-x8lgr"]
Jul 02 19:19:04.711399 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.710994    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-8fd569cc8-x8lgr" podUID="6d91bfaa-f50b-4b2c-b824-53c33a4c3618" containerName="manager" containerID="cri-o://c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9" gracePeriod=30
Jul 02 19:19:04.964598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.964540    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:19:04.978953 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.978935    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-webhook-cert\") pod \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") "
Jul 02 19:19:04.979055 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.978997    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5cmw\" (UniqueName: \"kubernetes.io/projected/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-kube-api-access-s5cmw\") pod \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\" (UID: \"6d91bfaa-f50b-4b2c-b824-53c33a4c3618\") "
Jul 02 19:19:04.981357 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.981326    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6d91bfaa-f50b-4b2c-b824-53c33a4c3618" (UID: "6d91bfaa-f50b-4b2c-b824-53c33a4c3618"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:19:04.981357 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:04.981337    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-kube-api-access-s5cmw" (OuterVolumeSpecName: "kube-api-access-s5cmw") pod "6d91bfaa-f50b-4b2c-b824-53c33a4c3618" (UID: "6d91bfaa-f50b-4b2c-b824-53c33a4c3618"). InnerVolumeSpecName "kube-api-access-s5cmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:19:05.079883 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.079849    2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-webhook-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:19:05.079883 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.079878    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5cmw\" (UniqueName: \"kubernetes.io/projected/6d91bfaa-f50b-4b2c-b824-53c33a4c3618-kube-api-access-s5cmw\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:19:05.806302 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.806267    2585 generic.go:356] "Generic (PLEG): container finished" podID="6d91bfaa-f50b-4b2c-b824-53c33a4c3618" containerID="c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9" exitCode=0
Jul 02 19:19:05.806711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.806316    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8fd569cc8-x8lgr"
Jul 02 19:19:05.806711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.806338    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8fd569cc8-x8lgr" event={"ID":"6d91bfaa-f50b-4b2c-b824-53c33a4c3618","Type":"ContainerDied","Data":"c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9"}
Jul 02 19:19:05.806711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.806390    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8fd569cc8-x8lgr" event={"ID":"6d91bfaa-f50b-4b2c-b824-53c33a4c3618","Type":"ContainerDied","Data":"d16cf28c0dcf5a50a4642a9cdeddf73be96515d5676768acbcffe87960e338ae"}
Jul 02 19:19:05.806711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.806420    2585 scope.go:117] "RemoveContainer" containerID="c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9"
Jul 02 19:19:05.814944 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.814925    2585 scope.go:117] "RemoveContainer" containerID="c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9"
Jul 02 19:19:05.815248 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:19:05.815225    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9\": container with ID starting with c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9 not found: ID does not exist" containerID="c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9"
Jul 02 19:19:05.815366 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.815255    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9"} err="failed to get container status \"c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9\": rpc error: code = NotFound desc = could not find container \"c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9\": container with ID starting with c08909b20241811f8272dcfc9403dc4dc5b610bde568328e6b1fd101ae9845a9 not found: ID does not exist"
Jul 02 19:19:05.825873 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.825846    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-x8lgr"]
Jul 02 19:19:05.828468 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:05.828446    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-x8lgr"]
Jul 02 19:19:06.784666 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.784623    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8fd569cc8-rvzcl"]
Jul 02 19:19:06.785123 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.785105    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d91bfaa-f50b-4b2c-b824-53c33a4c3618" containerName="manager"
Jul 02 19:19:06.785237 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.785124    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d91bfaa-f50b-4b2c-b824-53c33a4c3618" containerName="manager"
Jul 02 19:19:06.785237 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.785225    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d91bfaa-f50b-4b2c-b824-53c33a4c3618" containerName="manager"
Jul 02 19:19:06.789891 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.789868    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:06.792317 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.792294    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef575828-0a7b-4bf4-9420-2d9c8644954c-webhook-cert\") pod \"maas-controller-8fd569cc8-rvzcl\" (UID: \"ef575828-0a7b-4bf4-9420-2d9c8644954c\") " pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:06.792404 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.792334    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g86q8\" (UniqueName: \"kubernetes.io/projected/ef575828-0a7b-4bf4-9420-2d9c8644954c-kube-api-access-g86q8\") pod \"maas-controller-8fd569cc8-rvzcl\" (UID: \"ef575828-0a7b-4bf4-9420-2d9c8644954c\") " pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:06.793871 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.793854    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5jqvs\""
Jul 02 19:19:06.793871 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.793868    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 02 19:19:06.793994 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.793871    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 02 19:19:06.798915 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.798894    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-rvzcl"]
Jul 02 19:19:06.892862 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.892830    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef575828-0a7b-4bf4-9420-2d9c8644954c-webhook-cert\") pod \"maas-controller-8fd569cc8-rvzcl\" (UID: \"ef575828-0a7b-4bf4-9420-2d9c8644954c\") " pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:06.893341 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.892878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g86q8\" (UniqueName: \"kubernetes.io/projected/ef575828-0a7b-4bf4-9420-2d9c8644954c-kube-api-access-g86q8\") pod \"maas-controller-8fd569cc8-rvzcl\" (UID: \"ef575828-0a7b-4bf4-9420-2d9c8644954c\") " pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:06.895394 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.895363    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef575828-0a7b-4bf4-9420-2d9c8644954c-webhook-cert\") pod \"maas-controller-8fd569cc8-rvzcl\" (UID: \"ef575828-0a7b-4bf4-9420-2d9c8644954c\") " pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:06.901196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:06.901177    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g86q8\" (UniqueName: \"kubernetes.io/projected/ef575828-0a7b-4bf4-9420-2d9c8644954c-kube-api-access-g86q8\") pod \"maas-controller-8fd569cc8-rvzcl\" (UID: \"ef575828-0a7b-4bf4-9420-2d9c8644954c\") " pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:07.101813 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:07.101710    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:07.236135 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:07.236106    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8fd569cc8-rvzcl"]
Jul 02 19:19:07.238241 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:19:07.238207    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef575828_0a7b_4bf4_9420_2d9c8644954c.slice/crio-ade1217cacb647257e9cf30d448fbf4f2f8f864ffe5000fd2620e5102b4daaf4 WatchSource:0}: Error finding container ade1217cacb647257e9cf30d448fbf4f2f8f864ffe5000fd2620e5102b4daaf4: Status 404 returned error can't find the container with id ade1217cacb647257e9cf30d448fbf4f2f8f864ffe5000fd2620e5102b4daaf4
Jul 02 19:19:07.315497 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:07.315468    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d91bfaa-f50b-4b2c-b824-53c33a4c3618" path="/var/lib/kubelet/pods/6d91bfaa-f50b-4b2c-b824-53c33a4c3618/volumes"
Jul 02 19:19:07.819038 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:07.819008    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8fd569cc8-rvzcl" event={"ID":"ef575828-0a7b-4bf4-9420-2d9c8644954c","Type":"ContainerStarted","Data":"ade1217cacb647257e9cf30d448fbf4f2f8f864ffe5000fd2620e5102b4daaf4"}
Jul 02 19:19:08.824916 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:08.824883    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8fd569cc8-rvzcl" event={"ID":"ef575828-0a7b-4bf4-9420-2d9c8644954c","Type":"ContainerStarted","Data":"d447d237de76dd467dfcc9181f8e3c2438205b09097005ba7bbc5ac7e4def6ff"}
Jul 02 19:19:08.825350 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:08.824958    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:19:08.842527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:08.842473    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8fd569cc8-rvzcl" podStartSLOduration=2.257682564 podStartE2EDuration="2.842457591s" totalImagesPullingTime="584.775027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:19:08.841713734 +0000 UTC m=+829.991745721" watchObservedRunningTime="2026-07-02 19:19:08.842457591 +0000 UTC m=+829.992489575"
Jul 02 19:19:19.835876 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:19:19.835843    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8fd569cc8-rvzcl"
Jul 02 19:31:08.441950 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.441916    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"]
Jul 02 19:31:08.445739 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.445719    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.448415 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.448391    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-4f4ba489-serving-cert\""
Jul 02 19:31:08.457388 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.457365    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"]
Jul 02 19:31:08.600807 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.600769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcae9007-696f-44de-8730-0126e7980b15-openshift-service-ca\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.600990 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.600839    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.600990 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.600914    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2vz2\" (UniqueName: \"kubernetes.io/projected/dcae9007-696f-44de-8730-0126e7980b15-kube-api-access-w2vz2\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.702293 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.702207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcae9007-696f-44de-8730-0126e7980b15-openshift-service-ca\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.702293 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.702273    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.702483 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.702301    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vz2\" (UniqueName: \"kubernetes.io/projected/dcae9007-696f-44de-8730-0126e7980b15-kube-api-access-w2vz2\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.702483 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:31:08.702373    2585 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-4f4ba489-serving-cert: secret "maas-api-e2e-ait-4f4ba489-serving-cert" not found
Jul 02 19:31:08.702483 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:31:08.702445    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls podName:dcae9007-696f-44de-8730-0126e7980b15 nodeName:}" failed. No retries permitted until 2026-07-02 19:31:09.202427514 +0000 UTC m=+1550.352459479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls") pod "maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" (UID: "dcae9007-696f-44de-8730-0126e7980b15") : secret "maas-api-e2e-ait-4f4ba489-serving-cert" not found
Jul 02 19:31:08.702988 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.702969    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcae9007-696f-44de-8730-0126e7980b15-openshift-service-ca\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:08.712007 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:08.711980    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2vz2\" (UniqueName: \"kubernetes.io/projected/dcae9007-696f-44de-8730-0126e7980b15-kube-api-access-w2vz2\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:09.207931 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:09.207892    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:09.210555 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:09.210522    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls\") pod \"maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") " pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:09.357337 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:09.357308    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:09.501584 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:09.501551    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"]
Jul 02 19:31:09.503194 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:31:09.503137    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcae9007_696f_44de_8730_0126e7980b15.slice/crio-a8da25492710a9a9b5936e2a6d29a48f51813762f690b07b26f1e3a84dd52d48 WatchSource:0}: Error finding container a8da25492710a9a9b5936e2a6d29a48f51813762f690b07b26f1e3a84dd52d48: Status 404 returned error can't find the container with id a8da25492710a9a9b5936e2a6d29a48f51813762f690b07b26f1e3a84dd52d48
Jul 02 19:31:09.504514 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:09.504496    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:31:10.220312 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:10.220267    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" event={"ID":"dcae9007-696f-44de-8730-0126e7980b15","Type":"ContainerStarted","Data":"a8da25492710a9a9b5936e2a6d29a48f51813762f690b07b26f1e3a84dd52d48"}
Jul 02 19:31:11.228607 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:11.228565    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" event={"ID":"dcae9007-696f-44de-8730-0126e7980b15","Type":"ContainerStarted","Data":"bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9"}
Jul 02 19:31:11.229001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:11.228668    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:11.247995 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:11.247944    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" podStartSLOduration=2.326484839 podStartE2EDuration="3.247926853s" totalImagesPullingTime="921.442014ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:31:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:31:11.246461329 +0000 UTC m=+1552.396493313" watchObservedRunningTime="2026-07-02 19:31:11.247926853 +0000 UTC m=+1552.397958861"
Jul 02 19:31:13.737806 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:13.737775    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"]
Jul 02 19:31:13.738277 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:13.738051    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" podUID="dcae9007-696f-44de-8730-0126e7980b15" containerName="maas-api" containerID="cri-o://bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9" gracePeriod=30
Jul 02 19:31:13.992777 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:13.992712    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:14.058286 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.058252    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2vz2\" (UniqueName: \"kubernetes.io/projected/dcae9007-696f-44de-8730-0126e7980b15-kube-api-access-w2vz2\") pod \"dcae9007-696f-44de-8730-0126e7980b15\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") "
Jul 02 19:31:14.058468 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.058313    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcae9007-696f-44de-8730-0126e7980b15-openshift-service-ca\") pod \"dcae9007-696f-44de-8730-0126e7980b15\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") "
Jul 02 19:31:14.058468 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.058428    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls\") pod \"dcae9007-696f-44de-8730-0126e7980b15\" (UID: \"dcae9007-696f-44de-8730-0126e7980b15\") "
Jul 02 19:31:14.058702 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.058679    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcae9007-696f-44de-8730-0126e7980b15-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcae9007-696f-44de-8730-0126e7980b15" (UID: "dcae9007-696f-44de-8730-0126e7980b15"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:31:14.058821 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.058809    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcae9007-696f-44de-8730-0126e7980b15-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:14.060416 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.060391    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcae9007-696f-44de-8730-0126e7980b15" (UID: "dcae9007-696f-44de-8730-0126e7980b15"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:31:14.060515 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.060459    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcae9007-696f-44de-8730-0126e7980b15-kube-api-access-w2vz2" (OuterVolumeSpecName: "kube-api-access-w2vz2") pod "dcae9007-696f-44de-8730-0126e7980b15" (UID: "dcae9007-696f-44de-8730-0126e7980b15"). InnerVolumeSpecName "kube-api-access-w2vz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:31:14.160420 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.160382    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcae9007-696f-44de-8730-0126e7980b15-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:14.160420 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.160414    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2vz2\" (UniqueName: \"kubernetes.io/projected/dcae9007-696f-44de-8730-0126e7980b15-kube-api-access-w2vz2\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:14.243786 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.243698    2585 generic.go:356] "Generic (PLEG): container finished" podID="dcae9007-696f-44de-8730-0126e7980b15" containerID="bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9" exitCode=0
Jul 02 19:31:14.243786 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.243747    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"
Jul 02 19:31:14.244005 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.243749    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" event={"ID":"dcae9007-696f-44de-8730-0126e7980b15","Type":"ContainerDied","Data":"bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9"}
Jul 02 19:31:14.244005 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.243878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb" event={"ID":"dcae9007-696f-44de-8730-0126e7980b15","Type":"ContainerDied","Data":"a8da25492710a9a9b5936e2a6d29a48f51813762f690b07b26f1e3a84dd52d48"}
Jul 02 19:31:14.244005 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.243908    2585 scope.go:117] "RemoveContainer" containerID="bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9"
Jul 02 19:31:14.253077 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.253062    2585 scope.go:117] "RemoveContainer" containerID="bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9"
Jul 02 19:31:14.253370 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:31:14.253349    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9\": container with ID starting with bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9 not found: ID does not exist" containerID="bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9"
Jul 02 19:31:14.253452 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.253375    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9"} err="failed to get container status \"bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9\": rpc error: code = NotFound desc = could not find container \"bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9\": container with ID starting with bdc31352b3f59c340da1eec251a0b4ea595a4af338a187f18367b7411917dbf9 not found: ID does not exist"
Jul 02 19:31:14.266528 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.266505    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"]
Jul 02 19:31:14.271240 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:14.271219    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-4f4ba489-6468cc9845-c7jtb"]
Jul 02 19:31:15.316231 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:15.316201    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcae9007-696f-44de-8730-0126e7980b15" path="/var/lib/kubelet/pods/dcae9007-696f-44de-8730-0126e7980b15/volumes"
Jul 02 19:31:20.196708 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.196675    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"]
Jul 02 19:31:20.197087 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.197063    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcae9007-696f-44de-8730-0126e7980b15" containerName="maas-api"
Jul 02 19:31:20.197087 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.197074    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcae9007-696f-44de-8730-0126e7980b15" containerName="maas-api"
Jul 02 19:31:20.197195 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.197130    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcae9007-696f-44de-8730-0126e7980b15" containerName="maas-api"
Jul 02 19:31:20.200008 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.199993    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.202868 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.202852    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-038ccab4-openshift-default-dockercfg-dqx5k\""
Jul 02 19:31:20.215662 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.215641    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"]
Jul 02 19:31:20.318795 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.318762    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-podinfo\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.318795 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.318803    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istiod-ca-cert\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.318883    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rpfd\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-kube-api-access-7rpfd\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.318927    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-token\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.318961    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-credential-socket\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.318992    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-certs\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319061 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.319054    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-socket\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.319106    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-envoy\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.319306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.319133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-data\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.419677 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.419632    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istiod-ca-cert\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.419879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.419695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rpfd\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-kube-api-access-7rpfd\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.419879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.419721    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-token\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.419879 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.419767    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-credential-socket\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420355 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420040    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-certs\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420355 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420320    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-socket\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420540 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420387    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-envoy\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420540 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420421    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-data\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420540 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420488    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-credential-socket\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420540 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420504    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-podinfo\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.420804 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420548    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-certs\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.427972 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420933    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istiod-ca-cert\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.427972 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.420946    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-socket\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.427972 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.421142    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-data\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.427972 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.423220    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-envoy\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.427972 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.423494    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-podinfo\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.430434 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.430409    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rpfd\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-kube-api-access-7rpfd\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.433067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.433045    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-token\") pod \"e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") " pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.511129 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.511048    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:20.648370 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.648343    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"]
Jul 02 19:31:20.651422 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:31:20.651387    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd534bcc2_3918_407d_b3c4_ec75a6c973e9.slice/crio-87c776c3ee285c409b1967a0b270069d000e3defec108cd800760e9755a77a15 WatchSource:0}: Error finding container 87c776c3ee285c409b1967a0b270069d000e3defec108cd800760e9755a77a15: Status 404 returned error can't find the container with id 87c776c3ee285c409b1967a0b270069d000e3defec108cd800760e9755a77a15
Jul 02 19:31:20.653822 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.653779    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:31:20.653911 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.653876    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:31:20.653948 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:20.653921    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 02 19:31:21.280563 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:21.280531    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" event={"ID":"d534bcc2-3918-407d-b3c4-ec75a6c973e9","Type":"ContainerStarted","Data":"c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07"}
Jul 02 19:31:21.281026 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:21.280581    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" event={"ID":"d534bcc2-3918-407d-b3c4-ec75a6c973e9","Type":"ContainerStarted","Data":"87c776c3ee285c409b1967a0b270069d000e3defec108cd800760e9755a77a15"}
Jul 02 19:31:21.305529 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:21.305479    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" podStartSLOduration=1.3054628529999999 podStartE2EDuration="1.305462853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:31:21.303278765 +0000 UTC m=+1562.453310769" watchObservedRunningTime="2026-07-02 19:31:21.305462853 +0000 UTC m=+1562.455494836"
Jul 02 19:31:21.511432 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:21.511400    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:21.516138 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:21.516114    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:22.285793 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:22.285762    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:22.286668 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:22.286648    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:32.928173 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:32.928120    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"]
Jul 02 19:31:32.928670 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:32.928414    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" podUID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" containerName="istio-proxy" containerID="cri-o://c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07" gracePeriod=30
Jul 02 19:31:37.288764 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:37.288721    2585 patch_prober.go:28] interesting pod/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 02 19:31:37.289267 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:37.288794    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" podUID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 19:31:38.187264 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.187230    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:38.297369 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297257    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-podinfo\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297369 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297345    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-envoy\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297369 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297376    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istiod-ca-cert\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297405    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-data\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297428    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-socket\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297610    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-credential-socket\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297684    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-token\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297703    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297714    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297715    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rpfd\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-kube-api-access-7rpfd\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297757    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297817    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-certs\") pod \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\" (UID: \"d534bcc2-3918-407d-b3c4-ec75a6c973e9\") "
Jul 02 19:31:38.297966 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.297901    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:31:38.298550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.298102    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:31:38.298550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.298273    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istiod-ca-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.298550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.298293    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.298550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.298305    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-socket\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.298550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.298319    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-credential-socket\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.298550 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.298332    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-workload-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.299626 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.299603    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 19:31:38.299721 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.299668    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:31:38.299721 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.299684    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-kube-api-access-7rpfd" (OuterVolumeSpecName: "kube-api-access-7rpfd") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "kube-api-access-7rpfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:31:38.299872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.299854    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "d534bcc2-3918-407d-b3c4-ec75a6c973e9" (UID: "d534bcc2-3918-407d-b3c4-ec75a6c973e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:31:38.360578 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.360545    2585 generic.go:356] "Generic (PLEG): container finished" podID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" containerID="c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07" exitCode=0
Jul 02 19:31:38.360742 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.360593    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"
Jul 02 19:31:38.360742 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.360634    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" event={"ID":"d534bcc2-3918-407d-b3c4-ec75a6c973e9","Type":"ContainerDied","Data":"c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07"}
Jul 02 19:31:38.360742 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.360690    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m" event={"ID":"d534bcc2-3918-407d-b3c4-ec75a6c973e9","Type":"ContainerDied","Data":"87c776c3ee285c409b1967a0b270069d000e3defec108cd800760e9755a77a15"}
Jul 02 19:31:38.360742 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.360718    2585 scope.go:117] "RemoveContainer" containerID="c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07"
Jul 02 19:31:38.370033 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.370016    2585 scope.go:117] "RemoveContainer" containerID="c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07"
Jul 02 19:31:38.370323 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:31:38.370303    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07\": container with ID starting with c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07 not found: ID does not exist" containerID="c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07"
Jul 02 19:31:38.370405 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.370331    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07"} err="failed to get container status \"c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07\": rpc error: code = NotFound desc = could not find container \"c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07\": container with ID starting with c4b35743ff61980f627badbb3fb9d981a6302f4ebfbfc85ed7d9afd7685d4b07 not found: ID does not exist"
Jul 02 19:31:38.399438 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.399398    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-podinfo\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.399438 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.399433    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-envoy\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.399438 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.399443    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-istio-token\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.399438 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.399451    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rpfd\" (UniqueName: \"kubernetes.io/projected/d534bcc2-3918-407d-b3c4-ec75a6c973e9-kube-api-access-7rpfd\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:38.408100 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.408071    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"]
Jul 02 19:31:38.412629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:38.412605    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-038ccab4-openshift-default-5bcc57df87-kxb9m"]
Jul 02 19:31:39.317782 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:39.317749    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" path="/var/lib/kubelet/pods/d534bcc2-3918-407d-b3c4-ec75a6c973e9/volumes"
Jul 02 19:31:40.778322 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.778289    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"]
Jul 02 19:31:40.778697 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.778683    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" containerName="istio-proxy"
Jul 02 19:31:40.778746 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.778698    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" containerName="istio-proxy"
Jul 02 19:31:40.778783 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.778763    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d534bcc2-3918-407d-b3c4-ec75a6c973e9" containerName="istio-proxy"
Jul 02 19:31:40.783332 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.783311    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:40.786593 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.786574    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-03f759d2-serving-cert\""
Jul 02 19:31:40.796626 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.796600    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"]
Jul 02 19:31:40.922407 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.922368    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a1ddf50-3179-401a-a692-db5dcc905d87-maas-api-tls\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:40.922407 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.922405    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ddf50-3179-401a-a692-db5dcc905d87-openshift-service-ca\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:40.922656 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:40.922461    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzgqp\" (UniqueName: \"kubernetes.io/projected/5a1ddf50-3179-401a-a692-db5dcc905d87-kube-api-access-fzgqp\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.023784 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.023753    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a1ddf50-3179-401a-a692-db5dcc905d87-maas-api-tls\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.023957 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.023804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ddf50-3179-401a-a692-db5dcc905d87-openshift-service-ca\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.023957 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.023848    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzgqp\" (UniqueName: \"kubernetes.io/projected/5a1ddf50-3179-401a-a692-db5dcc905d87-kube-api-access-fzgqp\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.024533 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.024507    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ddf50-3179-401a-a692-db5dcc905d87-openshift-service-ca\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.026346 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.026321    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a1ddf50-3179-401a-a692-db5dcc905d87-maas-api-tls\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.032803 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.032747    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzgqp\" (UniqueName: \"kubernetes.io/projected/5a1ddf50-3179-401a-a692-db5dcc905d87-kube-api-access-fzgqp\") pod \"maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") " pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.094009 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.093975    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:41.234040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.234015    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"]
Jul 02 19:31:41.236078 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:31:41.236056    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1ddf50_3179_401a_a692_db5dcc905d87.slice/crio-2e6888c6c782ceb6adda3904c366387d678ce2b68dcaa8e7f4c3568584394f79 WatchSource:0}: Error finding container 2e6888c6c782ceb6adda3904c366387d678ce2b68dcaa8e7f4c3568584394f79: Status 404 returned error can't find the container with id 2e6888c6c782ceb6adda3904c366387d678ce2b68dcaa8e7f4c3568584394f79
Jul 02 19:31:41.376036 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:41.375959    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8" event={"ID":"5a1ddf50-3179-401a-a692-db5dcc905d87","Type":"ContainerStarted","Data":"2e6888c6c782ceb6adda3904c366387d678ce2b68dcaa8e7f4c3568584394f79"}
Jul 02 19:31:42.388826 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:42.388790    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8" event={"ID":"5a1ddf50-3179-401a-a692-db5dcc905d87","Type":"ContainerStarted","Data":"15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4"}
Jul 02 19:31:42.389233 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:42.388887    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:42.411840 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:42.411786    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8" podStartSLOduration=2.015193346 podStartE2EDuration="2.411769637s" totalImagesPullingTime="396.576291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:31:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:31:42.410929071 +0000 UTC m=+1583.560961054" watchObservedRunningTime="2026-07-02 19:31:42.411769637 +0000 UTC m=+1583.561801619"
Jul 02 19:31:45.456556 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.456492    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"]
Jul 02 19:31:45.457024 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.456787    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8" podUID="5a1ddf50-3179-401a-a692-db5dcc905d87" containerName="maas-api" containerID="cri-o://15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4" gracePeriod=30
Jul 02 19:31:45.729257 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.729225    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:45.767040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.767007    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a1ddf50-3179-401a-a692-db5dcc905d87-maas-api-tls\") pod \"5a1ddf50-3179-401a-a692-db5dcc905d87\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") "
Jul 02 19:31:45.767252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.767049    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ddf50-3179-401a-a692-db5dcc905d87-openshift-service-ca\") pod \"5a1ddf50-3179-401a-a692-db5dcc905d87\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") "
Jul 02 19:31:45.767252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.767193    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzgqp\" (UniqueName: \"kubernetes.io/projected/5a1ddf50-3179-401a-a692-db5dcc905d87-kube-api-access-fzgqp\") pod \"5a1ddf50-3179-401a-a692-db5dcc905d87\" (UID: \"5a1ddf50-3179-401a-a692-db5dcc905d87\") "
Jul 02 19:31:45.767484 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.767457    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a1ddf50-3179-401a-a692-db5dcc905d87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a1ddf50-3179-401a-a692-db5dcc905d87" (UID: "5a1ddf50-3179-401a-a692-db5dcc905d87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:31:45.769065 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.769038    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a1ddf50-3179-401a-a692-db5dcc905d87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a1ddf50-3179-401a-a692-db5dcc905d87" (UID: "5a1ddf50-3179-401a-a692-db5dcc905d87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:31:45.769259 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.769232    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1ddf50-3179-401a-a692-db5dcc905d87-kube-api-access-fzgqp" (OuterVolumeSpecName: "kube-api-access-fzgqp") pod "5a1ddf50-3179-401a-a692-db5dcc905d87" (UID: "5a1ddf50-3179-401a-a692-db5dcc905d87"). InnerVolumeSpecName "kube-api-access-fzgqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:31:45.868579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.868543    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a1ddf50-3179-401a-a692-db5dcc905d87-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:45.868579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.868571    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a1ddf50-3179-401a-a692-db5dcc905d87-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:45.868579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:45.868581    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzgqp\" (UniqueName: \"kubernetes.io/projected/5a1ddf50-3179-401a-a692-db5dcc905d87-kube-api-access-fzgqp\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:31:46.408437 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.408404    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a1ddf50-3179-401a-a692-db5dcc905d87" containerID="15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4" exitCode=0
Jul 02 19:31:46.408633 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.408450    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"
Jul 02 19:31:46.408633 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.408488    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8" event={"ID":"5a1ddf50-3179-401a-a692-db5dcc905d87","Type":"ContainerDied","Data":"15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4"}
Jul 02 19:31:46.408633 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.408545    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8" event={"ID":"5a1ddf50-3179-401a-a692-db5dcc905d87","Type":"ContainerDied","Data":"2e6888c6c782ceb6adda3904c366387d678ce2b68dcaa8e7f4c3568584394f79"}
Jul 02 19:31:46.408633 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.408573    2585 scope.go:117] "RemoveContainer" containerID="15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4"
Jul 02 19:31:46.417477 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.417460    2585 scope.go:117] "RemoveContainer" containerID="15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4"
Jul 02 19:31:46.417748 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:31:46.417729    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4\": container with ID starting with 15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4 not found: ID does not exist" containerID="15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4"
Jul 02 19:31:46.417792 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.417758    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4"} err="failed to get container status \"15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4\": rpc error: code = NotFound desc = could not find container \"15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4\": container with ID starting with 15112f705c8bbfc50d68daec6bf49d9c6971605fe3671a42a96a50b3b8a960a4 not found: ID does not exist"
Jul 02 19:31:46.432094 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.432067    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"]
Jul 02 19:31:46.433770 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:46.433749    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-03f759d2-75fb58b7bd-k4gc8"]
Jul 02 19:31:47.316073 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:31:47.316044    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a1ddf50-3179-401a-a692-db5dcc905d87" path="/var/lib/kubelet/pods/5a1ddf50-3179-401a-a692-db5dcc905d87/volumes"
Jul 02 19:37:11.032450 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.032376    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"]
Jul 02 19:37:11.032880 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.032762    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a1ddf50-3179-401a-a692-db5dcc905d87" containerName="maas-api"
Jul 02 19:37:11.032880 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.032772    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a1ddf50-3179-401a-a692-db5dcc905d87" containerName="maas-api"
Jul 02 19:37:11.032880 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.032845    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a1ddf50-3179-401a-a692-db5dcc905d87" containerName="maas-api"
Jul 02 19:37:11.035896 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.035876    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.039872 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.039846    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-4ae35c-serving-cert\""
Jul 02 19:37:11.051715 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.051691    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"]
Jul 02 19:37:11.225056 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.225019    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f2d79-2a6a-4e73-9577-3698e69db221-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.225266 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.225087    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f2d79-2a6a-4e73-9577-3698e69db221-maas-api-tls\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.225266 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.225131    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9vz\" (UniqueName: \"kubernetes.io/projected/ea4f2d79-2a6a-4e73-9577-3698e69db221-kube-api-access-9v9vz\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.326594 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.326511    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f2d79-2a6a-4e73-9577-3698e69db221-maas-api-tls\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.326594 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.326553    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9vz\" (UniqueName: \"kubernetes.io/projected/ea4f2d79-2a6a-4e73-9577-3698e69db221-kube-api-access-9v9vz\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.326825 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.326638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f2d79-2a6a-4e73-9577-3698e69db221-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.327313 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.327290    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f2d79-2a6a-4e73-9577-3698e69db221-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.328983 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.328961    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f2d79-2a6a-4e73-9577-3698e69db221-maas-api-tls\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.336884 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.336854    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9vz\" (UniqueName: \"kubernetes.io/projected/ea4f2d79-2a6a-4e73-9577-3698e69db221-kube-api-access-9v9vz\") pod \"maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") " pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.347597 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.347577    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.489273 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.489248    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"]
Jul 02 19:37:11.491244 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:37:11.491217    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4f2d79_2a6a_4e73_9577_3698e69db221.slice/crio-7ab0ced5ccebf08d9798ccd8a9d252a98425431d3409fde892a68175d384df5d WatchSource:0}: Error finding container 7ab0ced5ccebf08d9798ccd8a9d252a98425431d3409fde892a68175d384df5d: Status 404 returned error can't find the container with id 7ab0ced5ccebf08d9798ccd8a9d252a98425431d3409fde892a68175d384df5d
Jul 02 19:37:11.492629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.492610    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:37:11.945390 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.945356    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl" event={"ID":"ea4f2d79-2a6a-4e73-9577-3698e69db221","Type":"ContainerStarted","Data":"116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21"}
Jul 02 19:37:11.945558 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.945409    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl" event={"ID":"ea4f2d79-2a6a-4e73-9577-3698e69db221","Type":"ContainerStarted","Data":"7ab0ced5ccebf08d9798ccd8a9d252a98425431d3409fde892a68175d384df5d"}
Jul 02 19:37:11.945558 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.945463    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:11.971908 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:11.971860    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl" podStartSLOduration=0.639165656 podStartE2EDuration="971.842547ms" totalImagesPullingTime="332.676891ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:37:11.969895327 +0000 UTC m=+1913.119927310" watchObservedRunningTime="2026-07-02 19:37:11.971842547 +0000 UTC m=+1913.121874850"
Jul 02 19:37:17.955661 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:17.955629    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:37:48.492459 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.492425    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"]
Jul 02 19:37:48.496325 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.496304    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.500298 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.500272    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4ae35c\"/\"default-dockercfg-qpm2r\""
Jul 02 19:37:48.500432 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.500309    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4ae35c\"/\"test-model-4ae35c-kserve-self-signed-certs\""
Jul 02 19:37:48.500432 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.500359    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4ae35c\"/\"openshift-service-ca.crt\""
Jul 02 19:37:48.500432 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.500357    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4ae35c\"/\"kube-root-ca.crt\""
Jul 02 19:37:48.506709 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.506686    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"]
Jul 02 19:37:48.681461 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.681429    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-home\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.681655 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.681467    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42mzc\" (UniqueName: \"kubernetes.io/projected/a684e54e-6e7c-4d88-ab9c-2cae133a781c-kube-api-access-42mzc\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.681655 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.681497    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-model-cache\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.681655 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.681624    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tls-certs\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.681828 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.681739    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tmp-dir\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.681828 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.681772    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-dshm\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783238    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tls-certs\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783323 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783307    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tmp-dir\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783538 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783325    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-dshm\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783538 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783348    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-home\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783538 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783368    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42mzc\" (UniqueName: \"kubernetes.io/projected/a684e54e-6e7c-4d88-ab9c-2cae133a781c-kube-api-access-42mzc\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783538 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783396    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-model-cache\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783853 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783823    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tmp-dir\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783973 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783872    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-model-cache\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.783973 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.783893    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-home\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.785702 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.785679    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-dshm\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.785821 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.785804    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tls-certs\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.791241 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.791221    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42mzc\" (UniqueName: \"kubernetes.io/projected/a684e54e-6e7c-4d88-ab9c-2cae133a781c-kube-api-access-42mzc\") pod \"test-model-4ae35c-kserve-78bbdcd9dc-vdhvb\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") " pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.806273 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.806250    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:48.947617 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:48.947592    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"]
Jul 02 19:37:48.949670 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:37:48.949637    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda684e54e_6e7c_4d88_ab9c_2cae133a781c.slice/crio-578c3ce243758148e04bd19d908046f2b1abbd536bf4334c430413cdab923862 WatchSource:0}: Error finding container 578c3ce243758148e04bd19d908046f2b1abbd536bf4334c430413cdab923862: Status 404 returned error can't find the container with id 578c3ce243758148e04bd19d908046f2b1abbd536bf4334c430413cdab923862
Jul 02 19:37:49.123776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:49.123695    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb" event={"ID":"a684e54e-6e7c-4d88-ab9c-2cae133a781c","Type":"ContainerStarted","Data":"43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a"}
Jul 02 19:37:49.123776 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:49.123742    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb" event={"ID":"a684e54e-6e7c-4d88-ab9c-2cae133a781c","Type":"ContainerStarted","Data":"578c3ce243758148e04bd19d908046f2b1abbd536bf4334c430413cdab923862"}
Jul 02 19:37:49.143228 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:49.143175    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb" podStartSLOduration=1.143160095 podStartE2EDuration="1.143160095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:37:49.142074505 +0000 UTC m=+1950.292106528" watchObservedRunningTime="2026-07-02 19:37:49.143160095 +0000 UTC m=+1950.293192070"
Jul 02 19:37:58.806823 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:58.806792    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:58.807207 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:58.806837    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:58.819202 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:58.819180    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:37:59.183657 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:37:59.183627    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:38:09.760732 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:09.760689    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"]
Jul 02 19:38:24.786782 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:24.786719    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb" podUID="a684e54e-6e7c-4d88-ab9c-2cae133a781c" containerName="main" containerID="cri-o://43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a" gracePeriod=45
Jul 02 19:38:25.033224 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.033198    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:38:25.134054 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.133973    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-dshm\") pod \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") "
Jul 02 19:38:25.134054 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134035    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tmp-dir\") pod \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") "
Jul 02 19:38:25.134265 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134067    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42mzc\" (UniqueName: \"kubernetes.io/projected/a684e54e-6e7c-4d88-ab9c-2cae133a781c-kube-api-access-42mzc\") pod \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") "
Jul 02 19:38:25.134265 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134145    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-home\") pod \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") "
Jul 02 19:38:25.134265 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134224    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-model-cache\") pod \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") "
Jul 02 19:38:25.134379 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134306    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tls-certs\") pod \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\" (UID: \"a684e54e-6e7c-4d88-ab9c-2cae133a781c\") "
Jul 02 19:38:25.134419 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134365    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a684e54e-6e7c-4d88-ab9c-2cae133a781c" (UID: "a684e54e-6e7c-4d88-ab9c-2cae133a781c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:25.134512 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134468    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-home" (OuterVolumeSpecName: "home") pod "a684e54e-6e7c-4d88-ab9c-2cae133a781c" (UID: "a684e54e-6e7c-4d88-ab9c-2cae133a781c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:25.134579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134545    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-model-cache" (OuterVolumeSpecName: "model-cache") pod "a684e54e-6e7c-4d88-ab9c-2cae133a781c" (UID: "a684e54e-6e7c-4d88-ab9c-2cae133a781c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:25.134649 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134633    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tmp-dir\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:25.134711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134649    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-home\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:25.134711 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.134659    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-model-cache\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:25.136349 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.136320    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-dshm" (OuterVolumeSpecName: "dshm") pod "a684e54e-6e7c-4d88-ab9c-2cae133a781c" (UID: "a684e54e-6e7c-4d88-ab9c-2cae133a781c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:38:25.136444 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.136347    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a684e54e-6e7c-4d88-ab9c-2cae133a781c-kube-api-access-42mzc" (OuterVolumeSpecName: "kube-api-access-42mzc") pod "a684e54e-6e7c-4d88-ab9c-2cae133a781c" (UID: "a684e54e-6e7c-4d88-ab9c-2cae133a781c"). InnerVolumeSpecName "kube-api-access-42mzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:38:25.136444 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.136357    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a684e54e-6e7c-4d88-ab9c-2cae133a781c" (UID: "a684e54e-6e7c-4d88-ab9c-2cae133a781c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:38:25.235792 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.235760    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a684e54e-6e7c-4d88-ab9c-2cae133a781c-tls-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:25.235792 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.235793    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a684e54e-6e7c-4d88-ab9c-2cae133a781c-dshm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:25.235953 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.235809    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42mzc\" (UniqueName: \"kubernetes.io/projected/a684e54e-6e7c-4d88-ab9c-2cae133a781c-kube-api-access-42mzc\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:25.306814 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.306786    2585 generic.go:356] "Generic (PLEG): container finished" podID="a684e54e-6e7c-4d88-ab9c-2cae133a781c" containerID="43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a" exitCode=0
Jul 02 19:38:25.306955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.306828    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"
Jul 02 19:38:25.306955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.306870    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb" event={"ID":"a684e54e-6e7c-4d88-ab9c-2cae133a781c","Type":"ContainerDied","Data":"43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a"}
Jul 02 19:38:25.306955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.306912    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb" event={"ID":"a684e54e-6e7c-4d88-ab9c-2cae133a781c","Type":"ContainerDied","Data":"578c3ce243758148e04bd19d908046f2b1abbd536bf4334c430413cdab923862"}
Jul 02 19:38:25.306955 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.306929    2585 scope.go:117] "RemoveContainer" containerID="43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a"
Jul 02 19:38:25.316185 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.316168    2585 scope.go:117] "RemoveContainer" containerID="43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a"
Jul 02 19:38:25.316479 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:38:25.316461    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a\": container with ID starting with 43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a not found: ID does not exist" containerID="43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a"
Jul 02 19:38:25.316527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.316487    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a"} err="failed to get container status \"43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a\": rpc error: code = NotFound desc = could not find container \"43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a\": container with ID starting with 43e3ab9721b69c1c1ba96a48009c7329282a0894ac8c46029859c8233b345f5a not found: ID does not exist"
Jul 02 19:38:25.329320 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.329298    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"]
Jul 02 19:38:25.333591 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:25.333571    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4ae35c/test-model-4ae35c-kserve-78bbdcd9dc-vdhvb"]
Jul 02 19:38:27.315673 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:27.315640    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a684e54e-6e7c-4d88-ab9c-2cae133a781c" path="/var/lib/kubelet/pods/a684e54e-6e7c-4d88-ab9c-2cae133a781c/volumes"
Jul 02 19:38:53.790227 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:53.790189    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"]
Jul 02 19:38:53.790716 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:53.790477    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl" podUID="ea4f2d79-2a6a-4e73-9577-3698e69db221" containerName="maas-api" containerID="cri-o://116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21" gracePeriod=30
Jul 02 19:38:54.042993 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.042940    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:38:54.108102 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.108068    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f2d79-2a6a-4e73-9577-3698e69db221-maas-api-tls\") pod \"ea4f2d79-2a6a-4e73-9577-3698e69db221\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") "
Jul 02 19:38:54.108324 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.108174    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f2d79-2a6a-4e73-9577-3698e69db221-openshift-service-ca\") pod \"ea4f2d79-2a6a-4e73-9577-3698e69db221\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") "
Jul 02 19:38:54.108324 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.108218    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v9vz\" (UniqueName: \"kubernetes.io/projected/ea4f2d79-2a6a-4e73-9577-3698e69db221-kube-api-access-9v9vz\") pod \"ea4f2d79-2a6a-4e73-9577-3698e69db221\" (UID: \"ea4f2d79-2a6a-4e73-9577-3698e69db221\") "
Jul 02 19:38:54.108518 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.108494    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4f2d79-2a6a-4e73-9577-3698e69db221-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea4f2d79-2a6a-4e73-9577-3698e69db221" (UID: "ea4f2d79-2a6a-4e73-9577-3698e69db221"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:38:54.110303 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.110273    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4f2d79-2a6a-4e73-9577-3698e69db221-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea4f2d79-2a6a-4e73-9577-3698e69db221" (UID: "ea4f2d79-2a6a-4e73-9577-3698e69db221"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:38:54.110303 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.110285    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4f2d79-2a6a-4e73-9577-3698e69db221-kube-api-access-9v9vz" (OuterVolumeSpecName: "kube-api-access-9v9vz") pod "ea4f2d79-2a6a-4e73-9577-3698e69db221" (UID: "ea4f2d79-2a6a-4e73-9577-3698e69db221"). InnerVolumeSpecName "kube-api-access-9v9vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:38:54.208941 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.208904    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f2d79-2a6a-4e73-9577-3698e69db221-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:54.208941 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.208935    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v9vz\" (UniqueName: \"kubernetes.io/projected/ea4f2d79-2a6a-4e73-9577-3698e69db221-kube-api-access-9v9vz\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:54.208941 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.208945    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f2d79-2a6a-4e73-9577-3698e69db221-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:38:54.450197 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.450141    2585 generic.go:356] "Generic (PLEG): container finished" podID="ea4f2d79-2a6a-4e73-9577-3698e69db221" containerID="116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21" exitCode=0
Jul 02 19:38:54.450364 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.450209    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"
Jul 02 19:38:54.450364 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.450221    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl" event={"ID":"ea4f2d79-2a6a-4e73-9577-3698e69db221","Type":"ContainerDied","Data":"116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21"}
Jul 02 19:38:54.450364 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.450263    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl" event={"ID":"ea4f2d79-2a6a-4e73-9577-3698e69db221","Type":"ContainerDied","Data":"7ab0ced5ccebf08d9798ccd8a9d252a98425431d3409fde892a68175d384df5d"}
Jul 02 19:38:54.450364 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.450283    2585 scope.go:117] "RemoveContainer" containerID="116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21"
Jul 02 19:38:54.459926 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.459904    2585 scope.go:117] "RemoveContainer" containerID="116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21"
Jul 02 19:38:54.460227 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:38:54.460182    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21\": container with ID starting with 116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21 not found: ID does not exist" containerID="116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21"
Jul 02 19:38:54.460227 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.460213    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21"} err="failed to get container status \"116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21\": rpc error: code = NotFound desc = could not find container \"116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21\": container with ID starting with 116f667409d171ff6836ec3ec6a6467404f43193effaebcaba5cc01177b04d21 not found: ID does not exist"
Jul 02 19:38:54.471092 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.471066    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"]
Jul 02 19:38:54.474969 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:54.474946    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-4ae35c-78bd94b496-dbrnl"]
Jul 02 19:38:55.322517 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:38:55.322483    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4f2d79-2a6a-4e73-9577-3698e69db221" path="/var/lib/kubelet/pods/ea4f2d79-2a6a-4e73-9577-3698e69db221/volumes"
Jul 02 19:39:03.532903 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.532862    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"]
Jul 02 19:39:03.533701 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.533674    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4f2d79-2a6a-4e73-9577-3698e69db221" containerName="maas-api"
Jul 02 19:39:03.533876 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.533861    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4f2d79-2a6a-4e73-9577-3698e69db221" containerName="maas-api"
Jul 02 19:39:03.534001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.533979    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a684e54e-6e7c-4d88-ab9c-2cae133a781c" containerName="main"
Jul 02 19:39:03.534001 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.534000    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a684e54e-6e7c-4d88-ab9c-2cae133a781c" containerName="main"
Jul 02 19:39:03.534163 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.534115    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4f2d79-2a6a-4e73-9577-3698e69db221" containerName="maas-api"
Jul 02 19:39:03.534163 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.534127    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a684e54e-6e7c-4d88-ab9c-2cae133a781c" containerName="main"
Jul 02 19:39:03.538970 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.538949    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.541748 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.541728    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-c8695a-serving-cert\""
Jul 02 19:39:03.548912 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.548889    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"]
Jul 02 19:39:03.597238 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.597211    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhdkj\" (UniqueName: \"kubernetes.io/projected/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-kube-api-access-hhdkj\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.597396 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.597300    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-openshift-service-ca\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.597396 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.597344    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.697803 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.697768    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.697984 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.697807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhdkj\" (UniqueName: \"kubernetes.io/projected/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-kube-api-access-hhdkj\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.697984 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.697877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-openshift-service-ca\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.697984 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:39:03.697923    2585 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-c8695a-serving-cert: secret "maas-api-e2e-api-a-c8695a-serving-cert" not found
Jul 02 19:39:03.698141 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:39:03.697991    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls podName:49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee nodeName:}" failed. No retries permitted until 2026-07-02 19:39:04.197968711 +0000 UTC m=+2025.348000692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls") pod "maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" (UID: "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee") : secret "maas-api-e2e-api-a-c8695a-serving-cert" not found
Jul 02 19:39:03.698849 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.698830    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-openshift-service-ca\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:03.708505 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:03.708474    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhdkj\" (UniqueName: \"kubernetes.io/projected/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-kube-api-access-hhdkj\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:04.203624 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:04.203582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:04.206007 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:04.205982    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls\") pod \"maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") " pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:04.451196 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:04.451135    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:04.588412 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:04.588381    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"]
Jul 02 19:39:04.591856 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:39:04.591820    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a20ecd_ce2d_4ef9_bb5b_ea3059dacbee.slice/crio-16e2502d13e774a3f0293fd5a3f325803d564025c023463054626313783b8d4a WatchSource:0}: Error finding container 16e2502d13e774a3f0293fd5a3f325803d564025c023463054626313783b8d4a: Status 404 returned error can't find the container with id 16e2502d13e774a3f0293fd5a3f325803d564025c023463054626313783b8d4a
Jul 02 19:39:05.505167 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:05.505107    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" event={"ID":"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee","Type":"ContainerStarted","Data":"517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22"}
Jul 02 19:39:05.505373 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:05.505180    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" event={"ID":"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee","Type":"ContainerStarted","Data":"16e2502d13e774a3f0293fd5a3f325803d564025c023463054626313783b8d4a"}
Jul 02 19:39:05.505373 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:05.505211    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:05.525552 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:05.525496    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" podStartSLOduration=2.010547072 podStartE2EDuration="2.525481458s" totalImagesPullingTime="514.934386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:39:05.523929462 +0000 UTC m=+2026.673961447" watchObservedRunningTime="2026-07-02 19:39:05.525481458 +0000 UTC m=+2026.675513528"
Jul 02 19:39:11.516830 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:11.516796    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:24.869700 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:24.869660    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"]
Jul 02 19:39:24.870312 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:24.869919    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" podUID="49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" containerName="maas-api" containerID="cri-o://517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22" gracePeriod=30
Jul 02 19:39:25.125273 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.125213    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:25.196434 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.196403    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-openshift-service-ca\") pod \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") "
Jul 02 19:39:25.196434 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.196439    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls\") pod \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") "
Jul 02 19:39:25.196700 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.196505    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhdkj\" (UniqueName: \"kubernetes.io/projected/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-kube-api-access-hhdkj\") pod \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\" (UID: \"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee\") "
Jul 02 19:39:25.196809 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.196784    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" (UID: "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:39:25.198584 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.198552    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" (UID: "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:39:25.198763 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.198738    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-kube-api-access-hhdkj" (OuterVolumeSpecName: "kube-api-access-hhdkj") pod "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" (UID: "49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee"). InnerVolumeSpecName "kube-api-access-hhdkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:39:25.297768 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.297730    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:39:25.297768 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.297761    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:39:25.297768 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.297771    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhdkj\" (UniqueName: \"kubernetes.io/projected/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee-kube-api-access-hhdkj\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:39:25.609041 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.609007    2585 generic.go:356] "Generic (PLEG): container finished" podID="49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" containerID="517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22" exitCode=0
Jul 02 19:39:25.609252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.609053    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"
Jul 02 19:39:25.609252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.609094    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" event={"ID":"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee","Type":"ContainerDied","Data":"517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22"}
Jul 02 19:39:25.609252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.609166    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p" event={"ID":"49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee","Type":"ContainerDied","Data":"16e2502d13e774a3f0293fd5a3f325803d564025c023463054626313783b8d4a"}
Jul 02 19:39:25.609252 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.609196    2585 scope.go:117] "RemoveContainer" containerID="517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22"
Jul 02 19:39:25.617906 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.617888    2585 scope.go:117] "RemoveContainer" containerID="517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22"
Jul 02 19:39:25.618135 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:39:25.618117    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22\": container with ID starting with 517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22 not found: ID does not exist" containerID="517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22"
Jul 02 19:39:25.618215 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.618158    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22"} err="failed to get container status \"517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22\": rpc error: code = NotFound desc = could not find container \"517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22\": container with ID starting with 517c794419c912b2b854e13046800924ff81f47cf2b59348fb8362540a05ea22 not found: ID does not exist"
Jul 02 19:39:25.626918 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.626893    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"]
Jul 02 19:39:25.630597 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:25.630569    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c8695a-5db5b74bcf-jkg5p"]
Jul 02 19:39:27.315390 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:27.315354    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" path="/var/lib/kubelet/pods/49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee/volumes"
Jul 02 19:39:49.835229 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.835131    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"]
Jul 02 19:39:49.835672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.835580    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" containerName="maas-api"
Jul 02 19:39:49.835672 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.835595    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" containerName="maas-api"
Jul 02 19:39:49.835760 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.835679    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a20ecd-ce2d-4ef9-bb5b-ea3059dacbee" containerName="maas-api"
Jul 02 19:39:49.840107 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.840084    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:49.842631 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.842608    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-f80343-serving-cert\""
Jul 02 19:39:49.847225 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.847202    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"]
Jul 02 19:39:49.941318 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.941279    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:49.941509 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.941378    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:49.941509 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:49.941472    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdxdk\" (UniqueName: \"kubernetes.io/projected/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-kube-api-access-tdxdk\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.042217 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.042181    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdxdk\" (UniqueName: \"kubernetes.io/projected/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-kube-api-access-tdxdk\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.042403 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.042273    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.042403 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.042325    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.042513 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:39:50.042418    2585 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-f80343-serving-cert: secret "maas-api-e2e-shared-b-f80343-serving-cert" not found
Jul 02 19:39:50.042513 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:39:50.042485    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls podName:2439a945-f8de-4ded-9aa9-e8031f5b3c9c nodeName:}" failed. No retries permitted until 2026-07-02 19:39:50.54246975 +0000 UTC m=+2071.692501712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls") pod "maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" (UID: "2439a945-f8de-4ded-9aa9-e8031f5b3c9c") : secret "maas-api-e2e-shared-b-f80343-serving-cert" not found
Jul 02 19:39:50.042893 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.042873    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.051110 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.051083    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdxdk\" (UniqueName: \"kubernetes.io/projected/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-kube-api-access-tdxdk\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.547386 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.547350    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.549833 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.549804    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") " pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.753598 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.753566    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:50.885492 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:50.885466    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"]
Jul 02 19:39:50.888248 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:39:50.888211    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2439a945_f8de_4ded_9aa9_e8031f5b3c9c.slice/crio-a98d626772a1b7d60a2faef4d9b852da06312851e0bbc438f2fef279e2f6007d WatchSource:0}: Error finding container a98d626772a1b7d60a2faef4d9b852da06312851e0bbc438f2fef279e2f6007d: Status 404 returned error can't find the container with id a98d626772a1b7d60a2faef4d9b852da06312851e0bbc438f2fef279e2f6007d
Jul 02 19:39:51.760188 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:51.760130    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" event={"ID":"2439a945-f8de-4ded-9aa9-e8031f5b3c9c","Type":"ContainerStarted","Data":"dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be"}
Jul 02 19:39:51.760371 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:51.760211    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" event={"ID":"2439a945-f8de-4ded-9aa9-e8031f5b3c9c","Type":"ContainerStarted","Data":"a98d626772a1b7d60a2faef4d9b852da06312851e0bbc438f2fef279e2f6007d"}
Jul 02 19:39:51.760371 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:51.760240    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:39:51.779691 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:51.779638    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" podStartSLOduration=2.422300244 podStartE2EDuration="2.779622912s" totalImagesPullingTime="357.322668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:39:51.778202661 +0000 UTC m=+2072.928234645" watchObservedRunningTime="2026-07-02 19:39:51.779622912 +0000 UTC m=+2072.929654908"
Jul 02 19:39:57.770019 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:39:57.769988    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:43:03.780110 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.780026    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"]
Jul 02 19:43:03.783686 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.783663    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.787821 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.787797    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"default-dockercfg-nbl6m\""
Jul 02 19:43:03.787960 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.787820    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"openshift-service-ca.crt\""
Jul 02 19:43:03.787960 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.787819    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"kube-root-ca.crt\""
Jul 02 19:43:03.787960 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.787875    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"sub-test-model-4a101a-kserve-self-signed-certs\""
Jul 02 19:43:03.793929 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.793907    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"]
Jul 02 19:43:03.853356 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.853321    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-model-cache\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.853356 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.853358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e41cc440-77e2-42f5-b667-310916733188-tls-certs\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.853546 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.853424    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-tmp-dir\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.853546 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.853471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-dshm\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.853546 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.853513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjhtx\" (UniqueName: \"kubernetes.io/projected/e41cc440-77e2-42f5-b667-310916733188-kube-api-access-zjhtx\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.853648 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.853574    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-home\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954321 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954286    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjhtx\" (UniqueName: \"kubernetes.io/projected/e41cc440-77e2-42f5-b667-310916733188-kube-api-access-zjhtx\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954346    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-home\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954390    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-model-cache\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954406    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e41cc440-77e2-42f5-b667-310916733188-tls-certs\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954446    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-tmp-dir\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954511 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954484    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-dshm\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954800 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954765    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-home\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954860 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954836    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-model-cache\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.954946 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.954857    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-tmp-dir\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.956803 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.956781    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-dshm\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.956998 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.956981    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e41cc440-77e2-42f5-b667-310916733188-tls-certs\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:03.965019 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:03.964996    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjhtx\" (UniqueName: \"kubernetes.io/projected/e41cc440-77e2-42f5-b667-310916733188-kube-api-access-zjhtx\") pod \"sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") " pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:04.094491 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:04.094395    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:04.229596 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:04.229571    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"]
Jul 02 19:43:04.231681 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:43:04.231645    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode41cc440_77e2_42f5_b667_310916733188.slice/crio-4a5fd1a926aed4199ef174fac6b422c80906ffad170d41900c68408a5eeff14d WatchSource:0}: Error finding container 4a5fd1a926aed4199ef174fac6b422c80906ffad170d41900c68408a5eeff14d: Status 404 returned error can't find the container with id 4a5fd1a926aed4199ef174fac6b422c80906ffad170d41900c68408a5eeff14d
Jul 02 19:43:04.670430 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:04.670394    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w" event={"ID":"e41cc440-77e2-42f5-b667-310916733188","Type":"ContainerStarted","Data":"f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f"}
Jul 02 19:43:04.670430 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:04.670438    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w" event={"ID":"e41cc440-77e2-42f5-b667-310916733188","Type":"ContainerStarted","Data":"4a5fd1a926aed4199ef174fac6b422c80906ffad170d41900c68408a5eeff14d"}
Jul 02 19:43:04.687302 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:04.687258    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w" podStartSLOduration=1.6872418850000002 podStartE2EDuration="1.687241885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:43:04.686495149 +0000 UTC m=+2265.836527143" watchObservedRunningTime="2026-07-02 19:43:04.687241885 +0000 UTC m=+2265.837273867"
Jul 02 19:43:14.094913 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:14.094873    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:14.095478 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:14.094935    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:14.107703 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:14.107679    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:43:14.734255 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:43:14.734224    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:44:16.982988 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:16.982951    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"]
Jul 02 19:44:32.003716 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.003673    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w" podUID="e41cc440-77e2-42f5-b667-310916733188" containerName="main" containerID="cri-o://f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f" gracePeriod=45
Jul 02 19:44:32.262767 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.262709    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:44:32.395873 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.395838    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-dshm\") pod \"e41cc440-77e2-42f5-b667-310916733188\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") "
Jul 02 19:44:32.396043 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.395895    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-home\") pod \"e41cc440-77e2-42f5-b667-310916733188\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") "
Jul 02 19:44:32.396043 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396003    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-tmp-dir\") pod \"e41cc440-77e2-42f5-b667-310916733188\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") "
Jul 02 19:44:32.396043 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396029    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-model-cache\") pod \"e41cc440-77e2-42f5-b667-310916733188\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") "
Jul 02 19:44:32.396208 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396057    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e41cc440-77e2-42f5-b667-310916733188-tls-certs\") pod \"e41cc440-77e2-42f5-b667-310916733188\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") "
Jul 02 19:44:32.396208 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396082    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjhtx\" (UniqueName: \"kubernetes.io/projected/e41cc440-77e2-42f5-b667-310916733188-kube-api-access-zjhtx\") pod \"e41cc440-77e2-42f5-b667-310916733188\" (UID: \"e41cc440-77e2-42f5-b667-310916733188\") "
Jul 02 19:44:32.396325 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396203    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-home" (OuterVolumeSpecName: "home") pod "e41cc440-77e2-42f5-b667-310916733188" (UID: "e41cc440-77e2-42f5-b667-310916733188"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:32.396325 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396214    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e41cc440-77e2-42f5-b667-310916733188" (UID: "e41cc440-77e2-42f5-b667-310916733188"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:32.396448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396335    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-model-cache" (OuterVolumeSpecName: "model-cache") pod "e41cc440-77e2-42f5-b667-310916733188" (UID: "e41cc440-77e2-42f5-b667-310916733188"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:32.396515 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396497    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-tmp-dir\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:32.396555 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396524    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-model-cache\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:32.396555 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.396542    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-home\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:32.397931 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.397912    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-dshm" (OuterVolumeSpecName: "dshm") pod "e41cc440-77e2-42f5-b667-310916733188" (UID: "e41cc440-77e2-42f5-b667-310916733188"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:44:32.398074 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.398054    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41cc440-77e2-42f5-b667-310916733188-kube-api-access-zjhtx" (OuterVolumeSpecName: "kube-api-access-zjhtx") pod "e41cc440-77e2-42f5-b667-310916733188" (UID: "e41cc440-77e2-42f5-b667-310916733188"). InnerVolumeSpecName "kube-api-access-zjhtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:44:32.398125 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.398115    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e41cc440-77e2-42f5-b667-310916733188-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e41cc440-77e2-42f5-b667-310916733188" (UID: "e41cc440-77e2-42f5-b667-310916733188"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:44:32.497502 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.497465    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e41cc440-77e2-42f5-b667-310916733188-tls-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:32.497502 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.497495    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjhtx\" (UniqueName: \"kubernetes.io/projected/e41cc440-77e2-42f5-b667-310916733188-kube-api-access-zjhtx\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:32.497707 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:32.497511    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e41cc440-77e2-42f5-b667-310916733188-dshm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:44:33.093439 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.093410    2585 generic.go:356] "Generic (PLEG): container finished" podID="e41cc440-77e2-42f5-b667-310916733188" containerID="f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f" exitCode=0
Jul 02 19:44:33.093845 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.093450    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"
Jul 02 19:44:33.093845 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.093493    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w" event={"ID":"e41cc440-77e2-42f5-b667-310916733188","Type":"ContainerDied","Data":"f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f"}
Jul 02 19:44:33.093845 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.093569    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w" event={"ID":"e41cc440-77e2-42f5-b667-310916733188","Type":"ContainerDied","Data":"4a5fd1a926aed4199ef174fac6b422c80906ffad170d41900c68408a5eeff14d"}
Jul 02 19:44:33.093845 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.093587    2585 scope.go:117] "RemoveContainer" containerID="f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f"
Jul 02 19:44:33.102668 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.102652    2585 scope.go:117] "RemoveContainer" containerID="f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f"
Jul 02 19:44:33.102912 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:44:33.102893    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f\": container with ID starting with f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f not found: ID does not exist" containerID="f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f"
Jul 02 19:44:33.103002 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.102920    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f"} err="failed to get container status \"f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f\": rpc error: code = NotFound desc = could not find container \"f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f\": container with ID starting with f5cc900d59df666811af0144de4e9384c0b546ef4bb094c744980d361332fc1f not found: ID does not exist"
Jul 02 19:44:33.122229 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.122203    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"]
Jul 02 19:44:33.127395 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.127368    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/sub-test-model-4a101a-kserve-5f6b9996ff-wdv4w"]
Jul 02 19:44:33.314910 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:44:33.314880    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e41cc440-77e2-42f5-b667-310916733188" path="/var/lib/kubelet/pods/e41cc440-77e2-42f5-b667-310916733188/volumes"
Jul 02 19:46:22.832077 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:22.832038    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"]
Jul 02 19:46:22.832659 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:22.832576    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" podUID="2439a945-f8de-4ded-9aa9-e8031f5b3c9c" containerName="maas-api" containerID="cri-o://dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be" gracePeriod=30
Jul 02 19:46:23.084986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.084927    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:46:23.173414 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.173383    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-openshift-service-ca\") pod \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") "
Jul 02 19:46:23.173579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.173451    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls\") pod \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") "
Jul 02 19:46:23.173579 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.173473    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdxdk\" (UniqueName: \"kubernetes.io/projected/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-kube-api-access-tdxdk\") pod \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\" (UID: \"2439a945-f8de-4ded-9aa9-e8031f5b3c9c\") "
Jul 02 19:46:23.173753 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.173729    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2439a945-f8de-4ded-9aa9-e8031f5b3c9c" (UID: "2439a945-f8de-4ded-9aa9-e8031f5b3c9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:46:23.175544 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.175521    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-kube-api-access-tdxdk" (OuterVolumeSpecName: "kube-api-access-tdxdk") pod "2439a945-f8de-4ded-9aa9-e8031f5b3c9c" (UID: "2439a945-f8de-4ded-9aa9-e8031f5b3c9c"). InnerVolumeSpecName "kube-api-access-tdxdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:46:23.175664 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.175555    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2439a945-f8de-4ded-9aa9-e8031f5b3c9c" (UID: "2439a945-f8de-4ded-9aa9-e8031f5b3c9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:46:23.274837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.274807    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:46:23.274837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.274834    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdxdk\" (UniqueName: \"kubernetes.io/projected/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-kube-api-access-tdxdk\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:46:23.274837 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.274844    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2439a945-f8de-4ded-9aa9-e8031f5b3c9c-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Jul 02 19:46:23.639289 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.639257    2585 generic.go:356] "Generic (PLEG): container finished" podID="2439a945-f8de-4ded-9aa9-e8031f5b3c9c" containerID="dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be" exitCode=0
Jul 02 19:46:23.639451 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.639300    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"
Jul 02 19:46:23.639451 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.639342    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" event={"ID":"2439a945-f8de-4ded-9aa9-e8031f5b3c9c","Type":"ContainerDied","Data":"dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be"}
Jul 02 19:46:23.639451 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.639393    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9" event={"ID":"2439a945-f8de-4ded-9aa9-e8031f5b3c9c","Type":"ContainerDied","Data":"a98d626772a1b7d60a2faef4d9b852da06312851e0bbc438f2fef279e2f6007d"}
Jul 02 19:46:23.639451 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.639415    2585 scope.go:117] "RemoveContainer" containerID="dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be"
Jul 02 19:46:23.648204 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.648182    2585 scope.go:117] "RemoveContainer" containerID="dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be"
Jul 02 19:46:23.648450 ip-10-0-133-120 kubenswrapper[2585]: E0702 19:46:23.648433    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be\": container with ID starting with dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be not found: ID does not exist" containerID="dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be"
Jul 02 19:46:23.648499 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.648459    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be"} err="failed to get container status \"dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be\": rpc error: code = NotFound desc = could not find container \"dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be\": container with ID starting with dda2e537f17e5e0a1d515f79e62b2c1d9f5075c4de2fed0adecffc7e7acc20be not found: ID does not exist"
Jul 02 19:46:23.657700 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.657676    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"]
Jul 02 19:46:23.660883 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:23.660861    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f80343-798d98b4c7-pg2w9"]
Jul 02 19:46:25.315602 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:25.315562    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2439a945-f8de-4ded-9aa9-e8031f5b3c9c" path="/var/lib/kubelet/pods/2439a945-f8de-4ded-9aa9-e8031f5b3c9c/volumes"
Jul 02 19:46:34.148644 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:34.148617    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kp7dg_5c88087d-ec69-42b3-bcef-05fd3f8683d5/manager/0.log"
Jul 02 19:46:34.296035 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:34.296002    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-m57zn_3316bb71-40d7-45c4-a4de-d15de892d3d4/manager/0.log"
Jul 02 19:46:34.419136 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:34.419047    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-57b5b68ff8-shtlb_93804015-97b3-4685-9223-97788020b3d1/maas-api/0.log"
Jul 02 19:46:34.780696 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:34.780617    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8fd569cc8-rvzcl_ef575828-0a7b-4bf4-9420-2d9c8644954c/manager/0.log"
Jul 02 19:46:34.891063 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:34.891034    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-sb69l_f4f541b1-420f-457c-9d27-89062ab4ddcc/server/0.log"
Jul 02 19:46:35.010069 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:35.010038    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h4f6b_7a672aa1-157f-4759-831a-5467b13da7a4/manager/0.log"
Jul 02 19:46:35.238165 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:35.238125    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-clkfx_e8d14bbf-3425-4043-82cf-3f78e17fb1ef/copy-manifests/0.log"
Jul 02 19:46:35.246945 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:35.246924    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-clkfx_e8d14bbf-3425-4043-82cf-3f78e17fb1ef/manager/0.log"
Jul 02 19:46:35.502067 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:35.501986    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-wwczn_45fdd633-bf3a-4902-b929-0ed89c4c290c/postgres/0.log"
Jul 02 19:46:36.894268 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:36.894231    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6t5g9_32430e5d-8fa6-4a5e-8d9d-8008f2269dc9/manager/0.log"
Jul 02 19:46:37.001306 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:37.001278    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j84cp_8db751b7-4a83-4ba8-9f63-5c44535701cd/manager/0.log"
Jul 02 19:46:37.106865 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:37.106835    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pdmlw_6360c37a-f946-46e8-a1fd-6347e3064d4d/kuadrant-console-plugin/0.log"
Jul 02 19:46:37.234854 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:37.234779    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-tpjqf_55e88b02-564b-4045-bca3-d8f9986a196b/registry-server/0.log"
Jul 02 19:46:37.593191 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:37.593088    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jq5lr_bc010be6-4f46-4ab8-b97c-b8b3e4734ae1/manager/0.log"
Jul 02 19:46:37.936248 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:37.936218    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xxk8n_6abec698-bdf0-445a-ae12-2dd287f7e97b/istio-proxy/0.log"
Jul 02 19:46:38.279926 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:38.279836    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-w8wqk_aa395899-6874-4a10-8c6b-397e8b0f9b29/kube-auth-proxy/0.log"
Jul 02 19:46:38.402115 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:38.402082    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/1.log"
Jul 02 19:46:38.638264 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:38.638238    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qmgtf_c283369d-18b5-4752-9430-c50a4de4807f/payload-processing/0.log"
Jul 02 19:46:39.096871 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:39.096785    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-v9jdp_0f6e2d12-0136-42fa-b6f0-6e2abd39771b/main/0.log"
Jul 02 19:46:39.314818 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:39.314792    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-zfn5z_3a7667e5-7688-4e19-b5d7-19d74e785108/main/0.log"
Jul 02 19:46:39.552940 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:39.552909    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-s8b26_42c9e949-c18e-449d-80cc-6a467915ab79/main/0.log"
Jul 02 19:46:46.173282 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:46.173254    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fc92r_575a34d7-8a37-4869-8082-5383d427e35b/global-pull-secret-syncer/0.log"
Jul 02 19:46:46.347652 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:46.347624    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c6czg_e0ad0df9-07c9-4a65-87a3-ee31c024ca6a/konnectivity-agent/0.log"
Jul 02 19:46:46.435843 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:46.435767    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-120.ec2.internal_4ad4923ba204444f53390e622f229eba/haproxy/0.log"
Jul 02 19:46:51.668904 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:51.668872    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6t5g9_32430e5d-8fa6-4a5e-8d9d-8008f2269dc9/manager/0.log"
Jul 02 19:46:51.695260 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:51.695235    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j84cp_8db751b7-4a83-4ba8-9f63-5c44535701cd/manager/0.log"
Jul 02 19:46:51.722020 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:51.721998    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pdmlw_6360c37a-f946-46e8-a1fd-6347e3064d4d/kuadrant-console-plugin/0.log"
Jul 02 19:46:51.771403 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:51.771378    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-tpjqf_55e88b02-564b-4045-bca3-d8f9986a196b/registry-server/0.log"
Jul 02 19:46:52.013629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:52.013550    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jq5lr_bc010be6-4f46-4ab8-b97c-b8b3e4734ae1/manager/0.log"
Jul 02 19:46:54.015218 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.015188    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6b84dd98f9-clmdm_29cd2afc-b74c-469e-b131-0e561e552253/metrics-server/0.log"
Jul 02 19:46:54.160349 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.160320    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-99r8p_0ecadfce-a9e6-4e71-8964-d96b54d4a382/node-exporter/0.log"
Jul 02 19:46:54.180716 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.180650    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-99r8p_0ecadfce-a9e6-4e71-8964-d96b54d4a382/kube-rbac-proxy/0.log"
Jul 02 19:46:54.202003 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.201982    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-99r8p_0ecadfce-a9e6-4e71-8964-d96b54d4a382/init-textfile/0.log"
Jul 02 19:46:54.406308 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.406279    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/prometheus/0.log"
Jul 02 19:46:54.431243 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.431174    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/config-reloader/0.log"
Jul 02 19:46:54.454213 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.454188    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/thanos-sidecar/0.log"
Jul 02 19:46:54.476611 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.476591    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/kube-rbac-proxy-web/0.log"
Jul 02 19:46:54.500405 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.500386    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/kube-rbac-proxy/0.log"
Jul 02 19:46:54.523733 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.523709    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/kube-rbac-proxy-thanos/0.log"
Jul 02 19:46:54.549184 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:54.549143    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f4c7246-27bc-4c0d-83e3-488ce3e03d71/init-config-reloader/0.log"
Jul 02 19:46:56.610901 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.610866    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"]
Jul 02 19:46:56.611367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.611309    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2439a945-f8de-4ded-9aa9-e8031f5b3c9c" containerName="maas-api"
Jul 02 19:46:56.611367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.611322    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2439a945-f8de-4ded-9aa9-e8031f5b3c9c" containerName="maas-api"
Jul 02 19:46:56.611367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.611334    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e41cc440-77e2-42f5-b667-310916733188" containerName="main"
Jul 02 19:46:56.611367 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.611340    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e41cc440-77e2-42f5-b667-310916733188" containerName="main"
Jul 02 19:46:56.611562 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.611404    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2439a945-f8de-4ded-9aa9-e8031f5b3c9c" containerName="maas-api"
Jul 02 19:46:56.611562 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.611413    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e41cc440-77e2-42f5-b667-310916733188" containerName="main"
Jul 02 19:46:56.614590 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.614571    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.617242 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.617210    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqkdz\"/\"openshift-service-ca.crt\""
Jul 02 19:46:56.617368 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.617258    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qqkdz\"/\"default-dockercfg-865ch\""
Jul 02 19:46:56.618424 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.618404    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqkdz\"/\"kube-root-ca.crt\""
Jul 02 19:46:56.623706 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.623682    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"]
Jul 02 19:46:56.680189 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.680133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-sys\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.680370 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.680228    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nxx7\" (UniqueName: \"kubernetes.io/projected/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-kube-api-access-5nxx7\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.680370 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.680248    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-podres\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.680370 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.680325    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-proc\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.680370 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.680361    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-lib-modules\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781122 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781081    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-sys\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781336 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781180    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nxx7\" (UniqueName: \"kubernetes.io/projected/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-kube-api-access-5nxx7\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781336 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-podres\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781336 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781233    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-sys\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781336 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781333    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-podres\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781340    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-proc\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781368    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-proc\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781527 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781374    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-lib-modules\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.781629 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.781524    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-lib-modules\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.790040 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.790008    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nxx7\" (UniqueName: \"kubernetes.io/projected/9a51ee21-fb7f-4dc1-ab8b-f66291b9baff-kube-api-access-5nxx7\") pod \"perf-node-gather-daemonset-nzs8b\" (UID: \"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:56.927009 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:56.926958    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:57.057509 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:57.057486    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"]
Jul 02 19:46:57.059403 ip-10-0-133-120 kubenswrapper[2585]: W0702 19:46:57.059373    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9a51ee21_fb7f_4dc1_ab8b_f66291b9baff.slice/crio-694b0417e1abec2e38b35c4406ba26e9c10a8ba46188acd868641cf6a35987c3 WatchSource:0}: Error finding container 694b0417e1abec2e38b35c4406ba26e9c10a8ba46188acd868641cf6a35987c3: Status 404 returned error can't find the container with id 694b0417e1abec2e38b35c4406ba26e9c10a8ba46188acd868641cf6a35987c3
Jul 02 19:46:57.809201 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:57.809165    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b" event={"ID":"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff","Type":"ContainerStarted","Data":"b9431fbe0f4c6d5f2bc5558cc39a07c02183bf1c6c57f011a794a7545c525028"}
Jul 02 19:46:57.809676 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:57.809220    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b" event={"ID":"9a51ee21-fb7f-4dc1-ab8b-f66291b9baff","Type":"ContainerStarted","Data":"694b0417e1abec2e38b35c4406ba26e9c10a8ba46188acd868641cf6a35987c3"}
Jul 02 19:46:57.809676 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:57.809240    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:46:57.826000 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:57.825946    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b" podStartSLOduration=1.8259323950000002 podStartE2EDuration="1.825932395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:46:57.825556907 +0000 UTC m=+2498.975588892" watchObservedRunningTime="2026-07-02 19:46:57.825932395 +0000 UTC m=+2498.975964377"
Jul 02 19:46:58.868923 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:58.868892    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dmlds_99f84018-e5e1-4d3e-8b4d-7c96d34480d3/dns/0.log"
Jul 02 19:46:58.889270 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:58.889238    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dmlds_99f84018-e5e1-4d3e-8b4d-7c96d34480d3/kube-rbac-proxy/0.log"
Jul 02 19:46:58.966840 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:58.966812    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xztmn_bd214edd-b8f8-4814-accb-dd631b0f402d/dns-node-resolver/0.log"
Jul 02 19:46:59.512545 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:59.512510    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-548595b9dc-ct24l_6e74e3a7-c2b7-466e-9c57-c0e9126e4146/registry/0.log"
Jul 02 19:46:59.602220 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:46:59.602189    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kjg2x_26ec4b82-e24c-4d87-b690-91add3af40c4/node-ca/0.log"
Jul 02 19:47:00.528798 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:00.528768    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xxk8n_6abec698-bdf0-445a-ae12-2dd287f7e97b/istio-proxy/0.log"
Jul 02 19:47:00.713162 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:00.713134    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-w8wqk_aa395899-6874-4a10-8c6b-397e8b0f9b29/kube-auth-proxy/0.log"
Jul 02 19:47:00.745448 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:00.745418    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/0.log"
Jul 02 19:47:00.797922 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:00.797841    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/1.log"
Jul 02 19:47:00.906954 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:00.906928    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qmgtf_c283369d-18b5-4752-9430-c50a4de4807f/payload-processing/0.log"
Jul 02 19:47:01.669521 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:01.669485    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jwx7v_36604680-9a15-4463-aaa5-721782988b0a/serve-healthcheck-canary/0.log"
Jul 02 19:47:02.272573 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:02.272542    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hg6fm_4a6fd1fa-b6de-4952-b0e6-b5d4ef848373/kube-rbac-proxy/0.log"
Jul 02 19:47:02.294852 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:02.294824    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hg6fm_4a6fd1fa-b6de-4952-b0e6-b5d4ef848373/exporter/0.log"
Jul 02 19:47:02.317842 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:02.317819    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hg6fm_4a6fd1fa-b6de-4952-b0e6-b5d4ef848373/extractor/0.log"
Jul 02 19:47:03.823971 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:03.823944    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-nzs8b"
Jul 02 19:47:04.496020 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:04.495936    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kp7dg_5c88087d-ec69-42b3-bcef-05fd3f8683d5/manager/0.log"
Jul 02 19:47:04.728209 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:04.728173    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-m57zn_3316bb71-40d7-45c4-a4de-d15de892d3d4/manager/0.log"
Jul 02 19:47:04.774138 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:04.774056    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-57b5b68ff8-shtlb_93804015-97b3-4685-9223-97788020b3d1/maas-api/0.log"
Jul 02 19:47:04.979858 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:04.979817    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8fd569cc8-rvzcl_ef575828-0a7b-4bf4-9420-2d9c8644954c/manager/0.log"
Jul 02 19:47:05.000892 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:05.000864    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-sb69l_f4f541b1-420f-457c-9d27-89062ab4ddcc/server/0.log"
Jul 02 19:47:05.107873 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:05.107788    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h4f6b_7a672aa1-157f-4759-831a-5467b13da7a4/manager/0.log"
Jul 02 19:47:05.189768 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:05.189739    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-clkfx_e8d14bbf-3425-4043-82cf-3f78e17fb1ef/manager/0.log"
Jul 02 19:47:05.211621 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:05.211596    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-clkfx_e8d14bbf-3425-4043-82cf-3f78e17fb1ef/copy-manifests/0.log"
Jul 02 19:47:05.329665 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:05.329637    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-wwczn_45fdd633-bf3a-4902-b929-0ed89c4c290c/postgres/0.log"
Jul 02 19:47:06.772845 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:06.772814    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xxk8n_6abec698-bdf0-445a-ae12-2dd287f7e97b/istio-proxy/0.log"
Jul 02 19:47:06.912254 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:06.912224    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-w8wqk_aa395899-6874-4a10-8c6b-397e8b0f9b29/kube-auth-proxy/0.log"
Jul 02 19:47:06.934388 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:06.934357    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/0.log"
Jul 02 19:47:06.988314 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:06.988268    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/1.log"
Jul 02 19:47:07.076172 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:07.076045    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qmgtf_c283369d-18b5-4752-9430-c50a4de4807f/payload-processing/0.log"
Jul 02 19:47:08.307754 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:08.307652    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xxk8n_6abec698-bdf0-445a-ae12-2dd287f7e97b/istio-proxy/0.log"
Jul 02 19:47:08.467219 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:08.467190    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-w8wqk_aa395899-6874-4a10-8c6b-397e8b0f9b29/kube-auth-proxy/0.log"
Jul 02 19:47:08.486046 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:08.486013    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/0.log"
Jul 02 19:47:08.541052 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:08.541023    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-s6kxs_b4448b2b-d5f9-48e1-9c63-50f09ddbbd8c/istio-proxy/1.log"
Jul 02 19:47:08.632302 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:08.632233    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qmgtf_c283369d-18b5-4752-9430-c50a4de4807f/payload-processing/0.log"
Jul 02 19:47:13.197124 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.197092    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7hw4t_d31aa88e-4093-4102-915e-ccd7cbd19178/kube-multus/0.log"
Jul 02 19:47:13.391986 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.391955    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/kube-multus-additional-cni-plugins/0.log"
Jul 02 19:47:13.412812 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.412783    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/egress-router-binary-copy/0.log"
Jul 02 19:47:13.434086 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.434059    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/cni-plugins/0.log"
Jul 02 19:47:13.457031 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.456968    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/bond-cni-plugin/0.log"
Jul 02 19:47:13.477781 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.477758    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/routeoverride-cni/0.log"
Jul 02 19:47:13.498655 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.498630    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/whereabouts-cni-bincopy/0.log"
Jul 02 19:47:13.519385 ip-10-0-133-120 kubenswrapper[2585]: I0702 19:47:13.519362    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dft29_e9336630-072c-4de9-af56-044b1f64ecef/whereabouts-cni/0.log"